Electronic apparatus, method for controlling electronic apparatus, and program

Abstract

To provide an electronic apparatus that can detect the heart rate of a human body with good accuracy through transmission and reception of radio waves, a method for controlling an electronic apparatus, and a program.SOLUTION: An electronic apparatus comprises: a transmission unit that transmits a transmission wave; a reception unit that receives a reflected wave from a target of the transmission wave; and a signal processing unit that detects the distance position, direction, and speed of the target on the basis of conversion signals after performing Fourier transformation on beat signals of the transmission wave and reflected wave. The signal processing unit comprises an extraction unit that extracts heartbeat information of the target on the basis of the detected distance, direction, and speed of the target. The extraction unit extracts a signal component corresponding to vibration along with the heartbeat of the target from the conversion signals by using a plurality of window functions centered on a plurality of distance ranges, applies a learning classifier to the extracted signal component to select an optimal signal most appropriate as the heartbeat, acquires a value indicating the interval between peak times from the optimal signal, and extracts the heartbeat interval of the target on the basis of the dispersion and the central value of the value indicating the interval.SELECTED DRAWING: Figure 2

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2022-148632, filed on September 16, 2022, the entire disclosure of which is incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to an electronic device, a control method for an electronic device, and a program. [Background technology]

[0003] For example, in fields such as the automobile industry, technology for measuring the distance between a vehicle and a predetermined object has become increasingly important. In particular, in recent years, various RADAR (Radio Detecting and Ranging) technologies have been researched, which measure the distance between a vehicle and an object by transmitting radio waves such as millimeter waves and receiving the waves reflected by the object, such as an obstacle. The importance of such technology for measuring distance is expected to increase in the future along with the development of technologies for assisting drivers in driving and technologies related to autonomous driving, which automates driving partially or completely.

[0004] Various proposals have also been made regarding technologies for detecting the presence of a specific object by receiving a reflected wave of a transmitted radio wave reflected by the object. For example, Patent Document 1 proposes a device that can detect the presence of a person and their biological information by using microwaves. Also, for example, Patent Document 2 proposes a device that detects vital signs such as the frequency of a living body's breathing or heartbeat based on a reflected signal from microwave radar. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-71825 [Patent Document 2] Patent Publication No. 2021-32880 Summary of the Invention

[0006] An electronic device according to an embodiment includes: a transmitting unit that transmits a transmission wave; a receiving unit that receives a reflected wave of the transmitted wave from a target; a signal processing unit that detects the distance, position, direction, and velocity of the target based on a converted signal obtained by Fourier transforming beat signals of the transmitted wave and the reflected wave; Equipped with. The signal processing unit includes an extraction unit that extracts heartbeat information of the detected target based on the distance, direction, and speed of the target. The extraction unit extracting signal components corresponding to vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; applying a learning classifier to the extracted signal components to select a heart sound signal that is most appropriate as a heart sound; performing envelope processing on the most appropriate heart sound signal to select the most appropriate signal as a heartbeat; A value indicating the interval between peak times is obtained from the optimum signal, and the heartbeat interval of the target is extracted based on the variance and median value of the value indicating the interval.

[0007] A method for controlling an electronic device according to an embodiment includes: transmitting a transmission wave from a transmitting unit; receiving a reflected wave of the transmitted wave from a target; detecting a distance position, a direction, and a velocity of the target based on a converted signal obtained by performing a Fourier transform on beat signals of the transmitted wave and the reflected wave; extracting heartbeat information of the detected target based on the distance, direction, and speed of the target; Includes. In the step of extracting heartbeat information of the target, extracting signal components corresponding to vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; applying a learning classifier to the extracted signal components to select a heart sound signal that is most appropriate as a heart sound; performing envelope processing on the most appropriate heart sound signal to select the most appropriate signal as a heartbeat; A value indicating the interval between peak times is obtained from the optimum signal, and the heartbeat interval of the target is extracted based on the variance and median value of the value indicating the interval.

[0008] A program according to an embodiment includes: For electronic devices, transmitting a transmission wave from a transmitting unit; receiving a reflected wave of the transmitted wave from a target; detecting a distance position, a direction, and a velocity of the target based on a converted signal obtained by performing a Fourier transform on beat signals of the transmitted wave and the reflected wave; extracting heartbeat information of the detected target based on the distance, direction, and speed of the target; Execute the following. In the step of extracting heartbeat information of the target, extracting signal components corresponding to vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; applying a learning classifier to the extracted signal components to select a heart sound signal that is most appropriate as a heart sound; performing envelope processing on the most appropriate heart sound signal to select the most appropriate signal as a heartbeat; A value indicating the interval between peak times is obtained from the optimum signal, and the heartbeat interval of the target is extracted based on the variance and median value of the value indicating the interval. [Brief explanation of the drawings]

[0009] [Figure 1] 1A and 1B are diagrams illustrating a usage mode of an electronic device according to an embodiment. [Figure 2] FIG. 1 is a functional block diagram illustrating a schematic configuration of an electronic device according to an embodiment. [Figure 3] FIG. 2 is a diagram illustrating the configuration of a signal processed by an electronic device according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 5] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 6] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 7] 1A and 1B are diagrams illustrating an example of an antenna arrangement and an operation principle in an antenna array of an electronic device according to an embodiment. [Figure 8] 1A and 1B are diagrams illustrating examples of antenna arrangements in an antenna array of an electronic device according to an embodiment. [Figure 9] FIG. 1 is a diagram illustrating an example of signal processing by an electronic device according to an embodiment. [Figure 10] FIG. 1 is a diagram illustrating an example of signal processing by an electronic device according to an embodiment. [Figure 11] 10 is a flowchart illustrating a comparative example of the operation of the electronic device according to the embodiment. [Figure 12] 10 is a flowchart illustrating an operation of an electronic device according to an embodiment. [Figure 13] 10 is a flowchart illustrating an operation of an electronic device according to an embodiment. [Figure 14] FIG. 10 is a diagram illustrating a multi-window process performed by an electronic device according to an embodiment. [Figure 15] 10A and 10B are diagrams illustrating the relationship between ranks representing a target signal and a noise signal in singular value decomposition by an electronic device according to an embodiment. [Figure 16] FIG. 1 is a diagram conceptually illustrating multi-resolution analysis using a discrete wavelet transform performed by an electronic device according to an embodiment. [Figure 17] FIG. 10 is a diagram conceptually illustrating the process of classifying and identifying the best heart sound envelope by an electronic device according to an embodiment. [Figure 18] FIG. 10 is a diagram showing a result of one-dimensional processing performed by continuous wavelet transform in an electronic device according to an embodiment. [Figure 19] FIG. 1 is a diagram conceptually illustrating machine learning by an electronic device according to an embodiment. [Figure 20] 10A and 10B are diagrams illustrating an example of classification and identification of signals by an electronic device according to an embodiment. [Figure 21] FIG. 10 illustrates an example of mean absolute percentage error of the envelope of the best heart sound by an electronic device according to one embodiment. [Figure 22] FIG. 2 is a diagram showing an enlarged view of the envelopes of two heart sounds detected by an electronic device according to an embodiment. [Figure 23] FIG. 10 is a diagram showing a histogram of peak intervals of heart sounds detected by an electronic device according to an embodiment. [Figure 24] 10A and 10B are diagrams illustrating extraction of a heart sound envelope by an electronic device according to an embodiment. [Figure 25] 10A and 10B are diagrams illustrating DP matching for RRI estimation by an electronic device according to an embodiment. [Figure 26] FIG. 1 is a diagram showing a time-series waveform of a heart sound obtained by an electronic device according to an embodiment. [Figure 27] FIG. 1 is a diagram showing a time-series waveform of a heart sound obtained by an electronic device according to an embodiment. [Figure 28] FIG. 10 is a diagram showing a time-series waveform of an RRI obtained by an electronic device according to an embodiment. [Figure 29] FIG. 10 is a diagram illustrating the power spectral density of an RRI obtained by an electronic device according to an embodiment. [Figure 30] 1A and 1B are diagrams illustrating examples of antenna arrangements in an antenna array of an electronic device according to an embodiment. [Figure 31] FIG. 10 is a diagram illustrating a negative peak in the reconstruction error output of an autoencoder in processing by an electronic device according to an embodiment. [Figure 32]FIG. 10 is a diagram illustrating a negative peak in the reconstruction error output of an autoencoder in processing by an electronic device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] If weak vibrations such as the heartbeat of a human body or the like can be detected with high accuracy by transmitting and receiving radio waves such as millimeter waves, this is expected to be useful in a wide variety of fields. An object of the present disclosure is to provide an electronic device, a control method for an electronic device, and a program that can detect the heartbeat of a human body or the like with high accuracy by transmitting and receiving radio waves. According to one embodiment, an electronic device, a control method for an electronic device, and a program that can detect the heartbeat of a human body or the like with high accuracy by transmitting and receiving radio waves can be provided. Hereinafter, one embodiment will be described in detail with reference to the drawings.

[0011] In the present disclosure, an "electronic device" may refer to a device that is powered by electricity. Furthermore, a "user" may refer to a person (typically a human) or an animal that uses a system and / or an electronic device according to an embodiment. A user may include a person who monitors a target, such as a human, by using an electronic device according to an embodiment. Furthermore, a "target" may refer to a person (e.g., a human or an animal) that is monitored by an electronic device according to an embodiment. Furthermore, a user may include a target.

[0012] In this disclosure, a "heart beat" may refer to the beating of the heart, and a "beat" may refer to the rhythmic contraction of the heart.

[0013] In addition, in this disclosure, the term "heart rate" refers to the number of times the heart beats within a certain period of time. For example, the heart rate may be the number of beats per minute. When the heart pumps blood, pulsations occur in the arteries. Therefore, the number of times the arteries beat may be referred to as the pulse rate, or simply as pulses.

[0014] Furthermore, in the present disclosure, a "heart sound" may refer to the sound of the heart beating. That is, a heart sound may refer to the sound that occurs when the heart contracts and expands. Here, a "heart sound" may be composed of a first low, long sound resulting from ventricular muscle tension, mitral valve closure, the start of blood ejection into the arteries, and / or acceleration of blood flow, followed by a second high, short sound resulting from aortic valve closure and / or pulmonary valve closure.

[0015] In the present disclosure, heart sounds are not necessarily limited to physical sounds based on air vibrations, but may refer to the vibrations themselves caused by the heartbeat (pulsation). For example, in the present disclosure, heartbeats may imply a vibration source, and heart sounds may imply the vibrations themselves caused by the vibration source. Furthermore, in the present disclosure, heartbeats may also imply heart sounds depending on the situation.

[0016] Furthermore, in the present disclosure, heart sounds may refer to vibrations of the body that accompany the movement of the heart. Heart sounds may refer to, for example, vibrations of the entire body, or the head, neck, chest, throat, arms, legs, wrists, or other body parts. Heart sounds may refer to, for example, vibrations of the entire body, or the skin on the surface of the head, neck, chest, throat, arms, legs, wrists, or other body parts. Heart sounds may refer to, for example, vibrations of clothing, underwear, glasses, or other attachments worn by the subject that accompany vibrations of the entire body, or the skin on the surface of the head, neck, chest, throat, arms, legs, wrists, or other body parts. Heartbeats may refer to the beating of the heart itself. Beat-to-beat intervals, heart rate, etc. may be calculated from the movement of the heartbeat. In the present disclosure, heartbeats may be referred to as pulses.

[0017] An electronic device according to an embodiment can detect the heartbeat of a human or other target present in the vicinity of the electronic device. Therefore, the electronic device according to an embodiment may be used in specific facilities used by socially active individuals, such as companies, hospitals, nursing homes, schools, sports gyms, and care facilities. For example, in a company, it is extremely important to understand and / or manage the health of employees. Similarly, it is extremely important to understand and / or manage the health of patients and medical professionals in a hospital, and residents and staff in a nursing home. The electronic device according to an embodiment may be used in any facility where it is desirable to understand and / or manage the health of a target, without being limited to the aforementioned facilities such as companies, hospitals, and nursing homes. Such facilities may also include non-commercial facilities, such as a user's home. Furthermore, the electronic device according to an embodiment may be used not only indoors but also outdoors. For example, the electronic device according to an embodiment may be used inside a moving vehicle, such as a train, bus, or airplane, or at a station or platform. Furthermore, the electronic device according to one embodiment may be used in a moving object such as an automobile, an airplane, or a ship, a hotel, a user's home, a living room, a bathroom, a toilet, or a bedroom.

[0018] An electronic device according to an embodiment may be used, for example, in a nursing facility or the like, to detect or monitor the heartbeat of a subject, such as a person requiring nursing care or care. Furthermore, when an abnormality is detected in the heartbeat of a subject, such as a person requiring nursing care or care, the electronic device according to an embodiment may issue a predetermined warning to the subject and / or other persons. Therefore, the electronic device according to an embodiment may allow the subject and / or staff at a nursing facility or the like to recognize that an abnormality is detected in the pulse of a subject, such as a person requiring nursing care or care. On the other hand, when no abnormality is detected in the heartbeat of a subject, such as a person requiring nursing care or care, (e.g., recognized as normal), the electronic device according to an embodiment may notify the subject and / or other persons to that effect. Therefore, the electronic device according to an embodiment may allow the subject and / or staff at a nursing facility or the like to recognize that the pulse of a subject, such as a person requiring nursing care or care, is normal.

[0019] Furthermore, the electronic device according to an embodiment may detect the pulse of animals other than humans. As an example, the electronic device according to an embodiment described below will be described as detecting the pulse of a human using a sensor based on technology such as millimeter-wave radar.

[0020] An electronic device according to an embodiment may be installed on any stationary object or any mobile object. The electronic device according to an embodiment can transmit a transmission wave to the surroundings of the electronic device from a transmission antenna. The electronic device according to an embodiment can receive a reflected wave of the transmission wave from a reception antenna. At least one of the transmission antenna and the reception antenna may be provided in the electronic device, or may be provided in, for example, a radar sensor.

[0021] Hereinafter, as a typical example, an electronic device according to an embodiment will be described as being stationary. Meanwhile, the subject (human) whose pulse is detected by the electronic device according to an embodiment may be stationary, moving, or moving while stationary. The electronic device according to an embodiment can measure the distance between the electronic device and an object in a situation where the object around the electronic device may move, similar to a normal radar sensor. Furthermore, the electronic device according to an embodiment can measure the distance between the electronic device and an object even when both the electronic device and the object are stationary.

[0022] An electronic device according to an embodiment will be described in detail below with reference to the drawings. First, an example of object detection by the electronic device according to an embodiment will be described.

[0023] Fig. 1 is a diagram illustrating an example of a usage mode of an electronic device according to an embodiment. Fig. 1 shows an example of an electronic device having a sensor function and including a transmitting antenna and a receiving antenna according to an embodiment.

[0024] As shown in FIG. 1, an electronic device 1 according to an embodiment may include a transmitter and a receiver, which will be described later. As will be described later, the transmitter may include a transmitter antenna array 24. The receiver may include a receiver antenna array 31. Specific configurations of the electronic device 1, the transmitter, and the receiver will be described later. For ease of viewing, FIG. 1 illustrates the electronic device 1 including the transmitter antenna array 24 and the receiver antenna array 31. The electronic device 1 may also include at least one of the other functional units, such as at least a part of the signal processing unit 10 (FIG. 2) included in the electronic device 1, as appropriate. The electronic device 1 may also include at least one of the other functional units, such as at least a part of the signal processing unit 10 (FIG. 2), external to the electronic device 1. In FIG. 1, the electronic device 1 may be moving or may be stationary.

[0025] In the example shown in FIG. 1 , the electronic device 1 is shown in a simplified form, with a transmitter having a transmitting antenna array 24 and a receiver having a receiving antenna array 31. The electronic device 1 may include, for example, multiple transmitters and / or multiple receivers. The transmitter may include a transmitting antenna array 24 consisting of multiple transmitting antennas. The receiver may include a receiving antenna array 31 consisting of multiple receiving antennas. Here, the locations at which the transmitters and / or receivers are installed in the electronic device 1 are not limited to the locations shown in FIG. 1 , and may be other locations as appropriate. The number of transmitters and / or receivers may be any number greater than or equal to one, depending on various conditions (or requirements) such as the range and / or accuracy of heartbeat detection by the electronic device 1.

[0026] As will be described later, the electronic device 1 transmits electromagnetic waves as transmission waves from the transmitting antenna array 24. For example, if a predetermined object (e.g., the target 200 shown in FIG. 1 ) is present around the electronic device 1, at least a portion of the transmission waves transmitted from the electronic device 1 is reflected by the object and becomes a reflected wave. Then, by receiving such a reflected wave, for example, by the receiving antenna array 31 of the electronic device 1, the electronic device 1 can detect the object as a target.

[0027] The electronic device 1 including the transmitting antenna array 24 may typically be a RADAR (Radio Detecting and Ranging) sensor that transmits and receives radio waves. However, the electronic device 1 is not limited to a radar sensor. The electronic device 1 according to an embodiment may be a sensor based on, for example, a light wave-based LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) technology. Such sensors may include, for example, a patch antenna. Technologies such as RADAR and LIDAR are already known, so detailed descriptions may be appropriately simplified or omitted. Furthermore, the electronic device 1 according to an embodiment may be a sensor based on a technology that detects objects by transmitting and receiving, for example, sound waves or ultrasonic waves.

[0028] The electronic device 1 shown in FIG. 1 receives, from the receiving antenna array 31, reflected waves of transmitted waves transmitted from the transmitting antenna array 24. In this way, the electronic device 1 can detect a predetermined object 200 that exists within a predetermined distance from the electronic device 1 as a target. For example, as shown in FIG. 1, the electronic device 1 can measure the distance L between the electronic device 1 and the predetermined object 200. The electronic device 1 can also measure the relative speed between the electronic device 1 and the predetermined object 200. Furthermore, the electronic device 1 can also measure the direction (arrival angle θ) in which the reflected wave from the predetermined object 200 arrives at the electronic device 1.

[0029] In Fig. 1, the XY plane may be, for example, a plane substantially parallel to the ground surface. In this case, the positive direction of the Z axis shown in Fig. 1 may indicate a vertically upward direction. In Fig. 1, the electronic device 1 may be disposed on a plane parallel to the XY plane. Also, in Fig. 1, the target 200 may be, for example, standing on the ground surface substantially parallel to the XY plane.

[0030] Here, the target 200 may be, for example, a human being present around the electronic device 1. The target 200 may also be a non-human living thing, such as an animal present around the electronic device 1. As described above, the target 200 may be moving, stationary, or static. In the present disclosure, the object detected by the electronic device 1 includes not only inanimate objects such as any object, but also living things such as people, dogs, cats, horses, and other animals. The object detected by the electronic device 1 of the present disclosure may include targets detected using radar technology, including people, objects, and animals. In the present disclosure, targets may include people, objects, and animals. In the following description, it is assumed that an object such as the target 200 present around the electronic device 1 is a human being (or an animal). Hereinafter, the "target 200" may also be referred to as the "subject 200" as appropriate. In the present disclosure, the target may be the above-mentioned target 200.

[0031] 1, the ratio between the size of the electronic device 1 and the size of the target 200 does not necessarily represent the actual ratio. Also, in FIG. 1, the transmitting antenna array 24 of the transmitting unit and the receiving antenna array 31 of the receiving unit are shown installed outside the electronic device 1. However, in one embodiment, the transmitting antenna array 24 of the transmitting unit and / or the receiving antenna array 31 of the receiving unit may be installed at various positions on the electronic device 1. For example, in one embodiment, the transmitting antenna array 24 of the transmitting unit and / or the receiving antenna array 31 of the receiving unit may be installed inside the electronic device 1 so as not to be visible from the outside of the electronic device 1.

[0032] In the following, as a typical example, the transmitting antenna of the electronic device 1 will be described as transmitting radio waves in a frequency band such as millimeter waves (30 GHz or higher) or quasi-millimeter waves (for example, around 20 GHz to 30 GHz). On the other hand, the transmitting antenna of the electronic device 1 may transmit radio waves having a frequency bandwidth of 4 GHz, for example, 77 GHz to 81 GHz.

[0033] 2 is a functional block diagram schematically illustrating an example of the configuration of the electronic device 1 according to an embodiment. An example of the configuration of the electronic device 1 according to an embodiment will be described below.

[0034] When measuring distances and the like using millimeter-wave radar, frequency modulated continuous wave radar (hereinafter referred to as FMCW radar) is often used. FMCW radar generates a transmission signal by sweeping the frequency of the radio waves to be transmitted. Therefore, in a millimeter-wave FMCW radar using radio waves in the 79 GHz frequency band, for example, the frequency of the radio waves used has a frequency bandwidth of 4 GHz, such as 77 GHz to 81 GHz. Radar using the 79 GHz frequency band is characterized by a wider usable frequency bandwidth than other millimeter-wave / quasi-millimeter-wave radars, such as those in the 24 GHz, 60 GHz, and 76 GHz frequency bands. Hereinafter, such an embodiment will be described as an example.

[0035] The FMCW radar system used in the present disclosure may include an FCM (Fast-Chirp Modulation) system that transmits chirp signals at a shorter period than normal. The signals generated by the electronic device 1 are not limited to FMCW signals. The signals generated by the electronic device 1 may be signals of various systems other than FMCW. The transmission signal sequence stored in any storage unit may differ depending on these various systems. For example, in the case of the above-mentioned FMCW radar signal, signals whose frequency increases and decreases with each time sample may be used. Since known technologies can be applied as appropriate to the above-mentioned various systems, further detailed explanations will be omitted.

[0036] As shown in FIG. 2 , the electronic device 1 according to an embodiment includes a signal processing unit 10. The signal processing unit 10 may include a signal generation processing unit 11, a received signal processing unit 12, a heartbeat extraction unit 13, and a calculation unit 14. The heartbeat extraction unit 13 may, for example, execute a process of extracting a micro-Doppler component. The heartbeat extraction unit 13 may also execute a process of extracting an envelope of the heart sounds of the subject 200. The calculation unit 14 may, for example, execute a process of calculating the heartbeat interval (RRI) of the subject 200. The calculation unit 14 may also execute a process of calculating the heartbeat of the subject 200. The calculation unit 14 may also execute a process of calculating the heart rate validity (HRV) of the subject 200. In this case, the calculation unit 14 may execute a process of performing frequency analysis on the extracted time series data of the heartbeat interval of the subject 200. The calculation unit 14 may also execute a process of calculating the heartbeat variability of the subject 200 based on the frequency analysis of the time series data of the heartbeat interval. The signal generation processing unit 11, the received signal processing unit 12, the heartbeat extraction unit 13, and the calculation unit 14 will be described further below as appropriate. In the present disclosure, the heart sound may be, for example, a chest vibration waveform (see FIG. 20, etc.) directly observed by radar, or may be chest vibration. The heartbeat is the heartbeat itself. The heartbeat interval, heart rate, etc. may be calculated from the movement of the heartbeat. The heartbeat interval may be the time interval between one heartbeat and the next heartbeat.

[0037] Moreover, the electronic device 1 according to an embodiment includes a transmitting unit that includes a transmitting DAC 21, a transmitting circuit 22, a millimeter-wave transmitting circuit 23, and a transmitting antenna array 24. Moreover, the electronic device 1 according to an embodiment includes a receiving unit that includes a receiving antenna array 31, a mixer 32, a receiving circuit 33, and a receiving ADC 34. The electronic device 1 according to an embodiment may not include at least one of the functional units shown in FIG. 2, or may include functional units other than the functional units shown in FIG. 2. The electronic device 1 shown in FIG. 2 may be configured using a circuit that is basically configured similarly to a general radar that uses electromagnetic waves in the millimeter-wave band or the like. Meanwhile, in the electronic device 1 according to an embodiment, the signal processing by the signal processing unit 10 may include processing that differs from that of conventional general radar.

[0038] The signal processing unit 10 included in the electronic device 1 according to an embodiment can control the overall operation of the electronic device 1, including the control of each functional unit constituting the electronic device 1. In particular, the signal processing unit 10 performs various processes on signals handled by the electronic device 1. The signal processing unit 10 may include at least one processor, such as a central processing unit (CPU) or a digital signal processor (DSP), to provide control and processing capabilities for executing various functions. The signal processing unit 10 may be implemented as a single processor, several processors, or individual processors. The processor may be implemented as a single integrated circuit. An integrated circuit is also called an IC (integrated circuit). The processor may be implemented as multiple integrated circuits and discrete circuits connected to each other in a communicative manner. The processor may be implemented based on various other known technologies. In an embodiment, the signal processing unit 10 may be configured as, for example, a CPU (hardware) and a program (software) executed by the CPU. The signal processing unit 10 may also include a storage unit (memory) necessary for the operation of the signal processing unit 10, as appropriate.

[0039] The signal generation processing unit 11 of the signal processing unit 10 generates a signal to be transmitted from the electronic device 1. In the electronic device 1 according to one embodiment, the signal generation processing unit 11 may generate a transmission signal (transmission chirp signal) such as a chirp signal. In particular, the signal generation processing unit 11 may generate a signal whose frequency changes periodically and linearly (linear chirp signal). For example, the signal generation processing unit 11 may generate a chirp signal whose frequency periodically and linearly increases from 77 GHz to 81 GHz over time. Alternatively, the signal generation processing unit 11 may generate a signal whose frequency periodically and linearly increases (up-chirp) and decreases (down-chirp) from 77 GHz to 81 GHz over time. The signal generated by the signal generation processing unit 11 may be preset in the signal processing unit 10, for example. Alternatively, the signal generated by the signal generation processing unit 11 may be pre-stored in a storage unit in the signal processing unit 10, for example. Chirp signals used in technical fields such as radar are well known, and therefore a detailed description thereof will be appropriately simplified or omitted. The signal generated by the signal generating processing unit 11 is supplied to the transmitting DAC 21. For this reason, the signal generating processing unit 11 may be connected to the transmitting DAC 21.

[0040] The transmission DAC (digital-to-analog converter) 21 has a function of converting the digital signal supplied from the signal generation processing unit 11 into an analog signal. The transmission DAC 21 may be configured to include a general digital-to-analog converter. The signal converted into an analog signal by the transmission DAC 21 is supplied to the transmission circuit 22. For this reason, the transmission DAC 21 may be connected to the transmission circuit 22.

[0041] The transmission circuit 22 has a function of converting the analog signal converted by the transmission DAC 21 into an intermediate frequency (IF) band. The transmission circuit 22 may be configured to include a general IF band transmission circuit. The signal processed by the transmission circuit 22 is supplied to the millimeter wave transmission circuit 23. For this reason, the transmission circuit 22 may be connected to the millimeter wave transmission circuit 23.

[0042] The millimeter-wave transmission circuit 23 has the function of transmitting the signal processed by the transmission circuit 22 as a millimeter wave (RF wave). The millimeter-wave transmission circuit 23 may be configured to include a general millimeter-wave transmission circuit. The signal processed by the millimeter-wave transmission circuit 23 is supplied to the transmission antenna array 24. For this reason, the millimeter-wave transmission circuit 23 may be connected to the transmission antenna array 24. The signal processed by the millimeter-wave transmission circuit 23 is also supplied to the mixer 32. For this reason, the millimeter-wave transmission circuit 23 may also be connected to the mixer 32.

[0043] The transmitting antenna array 24 is an array of multiple transmitting antennas. In Fig. 2, the configuration of the transmitting antenna array 24 is shown in a simplified form. The transmitting antenna array 24 transmits the signal processed by the millimeter-wave transmitting circuit 23 to the outside of the electronic device 1. The transmitting antenna array 24 may be configured to include a transmitting antenna array used in a general millimeter-wave radar.

[0044] In this way, the electronic device 1 according to the embodiment includes a transmitting antenna (transmitting antenna array 24), and can transmit a transmitting signal (for example, a transmitting chirp signal) from the transmitting antenna array 24 as a transmitting wave.

[0045] 2, assume that an object such as a subject 200 is present around the electronic device 1. In this case, at least a portion of the transmission waves transmitted from the transmitting antenna array 24 is reflected by the object such as the subject 200. Of the transmission waves transmitted from the transmitting antenna array 24, at least a portion of those reflected by the object such as the subject 200 may be reflected toward the receiving antenna array 31.

[0046] The receiving antenna array 31 receives the reflected waves, which may be at least a portion of the transmitted waves transmitted from the transmitting antenna array 24 that are reflected by an object such as the subject 200.

[0047] The receiving antenna array 31 is an array of multiple receiving antennas. In FIG. 2, the configuration of the receiving antenna array 31 is shown in a simplified form. The receiving antenna array 31 receives reflected waves that are the result of reflection of the transmitted waves transmitted from the transmitting antenna array 24. The receiving antenna array 31 may be configured to include a receiving antenna array used in a general millimeter-wave radar. The receiving antenna array 31 supplies the received signals received as reflected waves to the mixer 32. For this reason, the receiving antenna array 31 may be connected to the mixer 32.

[0048] The mixer 32 converts the signal (transmission signal) processed by the millimeter-wave transmission circuit 23 and the reception signal received by the reception antenna array 31 into an intermediate frequency (IF) band. The mixer 32 may be configured to include a mixer used in a general millimeter-wave radar. The mixer 32 supplies the signal generated as a result of the combination to the reception circuit 33. For this reason, the mixer 32 may be connected to the reception circuit 33.

[0049] The receiving circuit 33 has a function of performing analog processing on the signal converted to the IF band by the mixer 32. The receiving circuit 33 may be configured to include a receiving circuit that converts to a general IF band. The signal processed by the receiving circuit 33 is supplied to the receiving ADC 34. For this reason, the receiving circuit 33 may be connected to the receiving ADC 34.

[0050] The receiving ADC (analog-to-digital converter) 34 has a function of converting the analog signal supplied from the receiving circuit 33 into a digital signal. The receiving ADC 34 may be configured to include a general analog-to-digital converter. The signal digitized by the receiving ADC 34 is supplied to the receiving signal processing unit 12 of the signal processing unit 10. For this reason, the receiving ADC 34 may be connected to the signal processing unit 10.

[0051] The reception signal processing unit 12 of the signal processing unit 10 has a function of performing various processes on the digital signal supplied from the reception DAC 34. For example, the reception signal processing unit 12 calculates the distance from the electronic device 1 to an object such as the subject 200 based on the digital signal supplied from the reception DAC 34 (distance measurement). The reception signal processing unit 12 also calculates the relative velocity of the object such as the subject 200 with respect to the electronic device 1 based on the digital signal supplied from the reception DAC 34 (velocity measurement). The reception signal processing unit 12 also calculates the azimuth angle of the object such as the subject 200 as seen from the electronic device 1 based on the digital signal supplied from the reception DAC 34 (angle measurement). Specifically, I / Q converted data may be input to the reception signal processing unit 12. By inputting such data, the reception signal processing unit 12 performs fast Fourier transforms (2D-FFT) in the range direction and the velocity direction, respectively. The received signal processing unit 12 then suppresses false alarms by removing noise points using processing such as CFAR (Constant False Alarm Rate) and makes the probability constant. The received signal processing unit 12 then estimates the angle of arrival for points that satisfy the CFAR criteria, thereby obtaining the position of an object such as the subject 200. Information generated as a result of measuring the distance, speed, and angle by the received signal processing unit 12 may be supplied to the heart rate extraction unit 13.

[0052] The heartbeat extraction unit 13 extracts information related to the heartbeat from the information generated by the received signal processing unit 12. The operation of extracting information related to the heartbeat by the heartbeat extraction unit 13 will be described further below. The information related to the heartbeat extracted by the heartbeat extraction unit 13 may be supplied to the calculation unit 14.

[0053] The calculation unit 14 performs various calculation processes and / or arithmetic processes on the information related to the heartbeat supplied from the heartbeat extraction unit 13. The various calculation processes and / or arithmetic processes performed by the calculation unit 14 will be described further below. The various pieces of information calculated and / or processed by the calculation unit 14 may be supplied to, for example, a communication interface 50. For this reason, the calculation unit 14 and / or the signal processing unit 10 may be connected to the communication interface 50. The various pieces of information calculated and / or processed by the calculation unit 14 may be supplied to functional units other than the communication interface 50.

[0054] The communication interface 50 includes an interface that outputs information supplied from the signal processing unit 10 to, for example, an external device 60. The communication interface 50 may output at least one of information regarding the position, velocity, and angle of an object such as the subject 200 to the external device 60 as a signal such as a CAN (Controller Area Network). For example, at least one of information regarding the position, velocity, and angle of an object such as the subject 200 may be supplied to the external device 60 via the communication interface 50. For this reason, the communication interface 50 may be connected to the external device 60.

[0055] 2, the electronic device 1 according to an embodiment may be connected to an external device 60 via a communication interface 50 in a wired or wireless manner. In an embodiment, the external device 60 may include any computer and / or any control device. The electronic device 1 according to an embodiment may also include the external device 60. The external device 60 may have various configurations depending on how the information on the heartbeat and / or heart sounds detected by the electronic device 1 is used. Therefore, a detailed description of the external device 60 will be omitted.

[0056] FIG. 3 is a diagram illustrating an example of a chirp signal generated by the signal generation processing unit 11 of the signal processing unit 10. In FIG.

[0057] FIG. 3 shows the time structure of one frame when using the FCM (Fast-Chirp Modulation) method. FIG. 3 shows an example of a received signal using the FCM method. FCM is a method in which chirp signals shown as c1, c2, c3, c4, ..., cn in FIG. 3 are repeated at short intervals (for example, equal to or longer than the round-trip time between the electromagnetic wave radar and the target, calculated from the maximum measured distance). In FCM, for convenience of signal processing of the received signal, transmission and reception processing is often performed by dividing the signal into subframe units as shown in FIG. 3.

[0058] In Fig. 3, the horizontal axis represents elapsed time, and the vertical axis represents frequency. In the example shown in Fig. 3, the signal generation processing unit 11 generates linear chirp signals whose frequencies change periodically and linearly. In Fig. 3, each chirp signal is represented as c1, c2, c3, c4, ..., cn. As shown in Fig. 3, the frequency of each chirp signal increases linearly with the passage of time.

[0059] In the example shown in FIG. 3, several chirp signals such as c1, c2, c3, c4, ..., cn are included in one subframe. That is, subframe 1 and subframe 2 shown in FIG. 3 are each configured to include several chirp signals such as c1, c2, c3, c4, ..., cn. Also, in the example shown in FIG. 3, several subframes such as subframe 1, subframe 2, ..., subframe N are included in one frame (1 frame). That is, 1 frame shown in FIG. 3 is configured to include N subframes. Also, 1 frame shown in FIG. 3 may be frame 1, followed by frame 2, frame 3, ..., etc. Each of these frames may be configured to include N subframes, just like frame 1. Also, a frame interval of a predetermined length may be included between frames. One frame shown in FIG. 3 may be, for example, 30 to 50 milliseconds long.

[0060] In the electronic device 1 according to one embodiment, the signal generation processing unit 11 may generate a transmission signal as any number of frames. Also, some chirp signals are omitted from the illustration in Fig. 3. In this manner, the relationship between time and frequency of the transmission signal generated by the signal generation processing unit 11 may be stored in, for example, a storage unit of the signal processing unit 10.

[0061] In this way, the electronic device 1 according to one embodiment may transmit a transmission signal consisting of subframes each including a plurality of chirp signals. Also, the electronic device 1 according to one embodiment may transmit a transmission signal consisting of a frame each including a predetermined number of subframes.

[0062] Hereinafter, the electronic device 1 will be described as transmitting a transmission signal having a frame structure as shown in FIG. 3. However, the frame structure as shown in FIG. 3 is merely an example, and for example, the number of chirp signals included in one subframe may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate subframes including any number of chirp signals (for example, any plural number). Also, the subframe structure as shown in FIG. 3 is merely an example, and for example, the number of subframes included in one frame may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate a frame including any number of subframes (for example, any plural number). The signal generation processing unit 11 may generate signals of different frequencies. The signal generation processing unit 11 may generate multiple discrete signals, each having a frequency f with a different bandwidth.

[0063] Fig. 4 is a diagram showing, in another aspect, part of the subframe shown in Fig. 3. Fig. 4 shows each sample of the received signal obtained by receiving the transmitted signal shown in Fig. 3 as a result of performing 2D-FFT (Two Dimensional Fast Fourier Transform), which is processing performed in the received signal processing unit 12 (Fig. 2) of the signal processing unit 10.

[0064] As shown in Fig. 4, chirp signals c1, c2, c3, c4, ..., cn are stored in each subframe, such as subframe 1, ..., subframe N. In Fig. 4, each chirp signal c1, c2, c3, c4, ..., cn is composed of samples, each represented by a square arranged in the horizontal direction. The received signal shown in Fig. 4 is subjected to 2D-FFT, CFAR, and / or integrated signal processing of each subframe by the received signal processing unit 12 shown in Fig. 2.

[0065] FIG. 5 is a diagram showing an example of a point group calculated on a range-Doppler (distance-velocity) plane as a result of 2D-FFT, CFAR, and integrated signal processing of each subframe being performed in the received signal processing unit 12 shown in FIG. 2.

[0066] In FIG. 5, the horizontal direction represents range (distance), and the vertical direction represents velocity. The filled squares s1 in FIG. 5 represent point clouds indicating signals that exceed the CFAR threshold processing. The unfilled squares s2 in FIG. 5 represent bins (2D-FFT samples) without point clouds that do not exceed the CFAR threshold. The point clouds on the range-Doppler plane calculated in FIG. 5 have their azimuth from the radar calculated by direction estimation, and the position and velocity on a two-dimensional plane are calculated as a point cloud indicating an object such as the subject 200. Here, the direction estimation may be calculated using a beamformer and / or a subspace method. Representative subspace method algorithms include MUSIC (MUltiple SIgnal Classification) and ESPRIT (Estimation of Signal Parameters via Rotation Invariance Technique).

[0067] Fig. 6 is a diagram showing an example of the result of the reception signal processing unit 12 converting the point cloud coordinates from the range-Doppler plane shown in Fig. 5 to the XY plane after performing direction estimation. As shown in Fig. 6, the reception signal processing unit 12 can plot the point cloud PG on the XY plane. Here, the point cloud PG is made up of points P. Furthermore, each point P has an angle θ and a radial velocity Vr in polar coordinates.

[0068] The received signal processing unit 12 detects an object present within the range where the transmitted wave T is transmitted, based on at least one of the results of the 2D-FFT and the angle estimation. The received signal processing unit 12 may perform object detection by, for example, clustering processing based on the estimated distance information, speed information, and angle information. Known algorithms used for clustering data include DBSCAN (Density-based spatial clustering of applications with noise). This is an algorithm that performs clustering based on density. In the clustering processing, for example, the average power of points constituting the detected object may be calculated. Information on the distance, speed, angle, and power of the object detected by the received signal processing unit 12 may be supplied to an external device 60, for example, via a communication interface 50.

[0069] As described above, the electronic device 1 may include a transmitting antenna (transmitting antenna array 24), a receiving antenna (receiving antenna array 31), and a signal processing unit 10. The transmitting antenna array 24 transmits a transmission wave T. The receiving antenna array 31 receives a reflected wave R resulting from reflection of the transmission wave T. The signal processing unit 10 then detects an object (such as the subject 200) that reflects the transmission wave T based on the transmission signal transmitted as the transmission wave T and the reception signal received as the reflected wave R.

[0070] Next, estimation of the direction of an incoming wave by the antenna array of the electronic device 1 according to an embodiment will be further described.

[0071] 7 is a diagram illustrating the configuration of the receiving antenna array 31 of the electronic device 1 according to one embodiment and the principle of estimating the direction of an incoming wave by the receiving antenna array 31. FIG. 7 shows an example of reception of radio waves by the receiving antenna array 31.

[0072] As shown in Figure 7, the receive antenna array 31 may be a linear arrangement of sensors such as receive antennas. As shown in Figure 7, in one embodiment, the receive antenna array 31 may include multiple receive antennas arranged in a linear arrangement. In Figure 7, the receive antenna array 31 includes antennas x1, x2, x3, ..., x M In the figure, multiple antennas such as those shown in the figure are represented by small circles. The receiving antenna array 31 may be composed of any number of antennas. As shown in FIG. 7, the multiple antennas constituting the receiving antenna array 31 are arranged at an array pitch d. A sensor array in which sensors (antennas, ultrasonic transducers, microphones, etc.) corresponding to various physical waves are arranged in an array is also called a uniform linear array (ULA). As shown in FIG. 7, physical waves (electromagnetic waves, sound waves, etc.) arrive from various directions, such as θ1 and θ2. Here, θ1 and θ2 may be the angles of arrival described above. In this way, a sensor array such as the receiving antenna array 31 can estimate the direction of arrival (angle of arrival) by utilizing the phase difference that occurs in the measurements between sensors depending on the direction of arrival of the physical wave. This method of estimating the direction of arrival of a wave is also referred to as angle of arrival estimation or direction of arrival (DoA).

[0073] In the electronic device 1 according to an embodiment, at least one of the transmitting antenna array 24 and the receiving antenna array 31 may be configured with multiple antennas arranged in a line. This allows, for example, millimeter-wave radar to appropriately narrow the directivity when transmitting and receiving radio waves. When transmitting a transmitted wave, the direction of the transmitted beam is often controlled by a beamformer. On the other hand, when receiving a reflected wave, the direction of arrival of the reflected wave is often estimated by a subspace method (such as the above-mentioned MUSIC and ESPRIT) rather than a beamformer. In the beamformer and subspace method, in a ULA such as that shown in FIG. 7, a phase difference occurs in the measurements between sensors depending on the direction of arrival of electromagnetic waves arriving from various directions. Therefore, the phase difference can be used to estimate the direction of arrival of the reflected wave.

[0074] Next, estimation of angles of incoming waves in two directions by the antenna array of the electronic device 1 according to one embodiment will be further described.

[0075] FIG. 8 is a diagram showing an example of an antenna arrangement for estimating the directions of arrival at two orthogonal angles.

[0076] As shown in FIG. 8, in the electronic device 1 according to one embodiment, the transmitting antenna array 24 and / or the receiving antenna array 31 may be configured to include an array of a plurality of patch antenna units.

[0077] In the transmitting antenna array 24 shown in FIG. 8, one patch antenna unit may be configured to include a plurality of elements electrically connected in the direction 1 shown in the figure. In each patch antenna unit, the plurality of elements may be electrically connected by wiring such as a stripline on a substrate. In each patch antenna unit, the plurality of elements are spaced apart at intervals d that are shorter than half the wavelength λ of the transmitting wave. 1,t 8, each patch antenna unit may have any number of elements greater than or equal to two electrically connected together.

[0078] 8, the transmitting antenna array 24 may be configured by arranging a plurality of patch antenna units in a direction 2 shown in the figure. The patch antenna units are spaced apart at intervals d, which are shorter than half the wavelength λ of the transmitting wave. 2,t In one embodiment, the transmit antenna array 24 may include any number of patch antenna units greater than or equal to two.

[0079] As shown in Fig. 8, in one embodiment, the receiving antenna array 31 may be configured by changing the arrangement of the multiple elements in the transmitting antenna array 24. That is, in the receiving antenna array 31 shown in Fig. 8, one patch antenna unit may be configured to include multiple elements electrically connected in the direction 2 shown in the figure. In each patch antenna unit, the multiple elements may be electrically connected by wiring such as a stripline on a substrate. In each patch antenna unit, the multiple elements are spaced apart at intervals d that are shorter than half the wavelength λ of the transmission wave. 2,s 8, each patch antenna unit may have any number of elements greater than or equal to two electrically connected together.

[0080] 8, the receiving antenna array 31 may be configured by arraying a plurality of patch antenna units in a direction 1 shown in the figure. The patch antenna units are spaced apart at intervals d, which are shorter than half the wavelength λ of the transmission wave. 1,s In one embodiment, the receive antenna array 31 may include any number of patch antenna units greater than or equal to two.

[0081] The elements included in the transmitting antenna array 24 and the receiving antenna array 31 may all be arranged on the same plane (for example, on the surface layer of the same substrate). The transmitting antenna array 24 and the receiving antenna array 31 may also be arranged close to each other (monostatic). Furthermore, directions 1 and 2 shown in FIG. 8 may be geometrically orthogonal to each other.

[0082] The transmitting antenna array 24 and the receiving antenna array 31 shown in FIG. 8 can appropriately narrow the directivity of each of the transmitting antennas and the receiving antennas. Furthermore, by using the transmitting antenna array 24 shown in FIG. 8 to control the direction of transmission of each transmission wave (transmission signal) at each timing of transmission, a beamformer for direction 2 shown in FIG. 8 can be realized. Furthermore, by using the receiving antenna array 31 shown in FIG. 8, the arrival direction of the reflected wave can be estimated for direction 1 shown in FIG. 8. In this way, it is possible to estimate the arrival direction of the reflected wave for two angles that are substantially orthogonal to each other. Therefore, it is possible to acquire a point cloud representing an object such as the subject 200 in three dimensions.

[0083] Next, a method for detecting the heartbeat of the subject 200 using the electronic device 1 according to an embodiment will be described.

[0084] An electronic device 1 according to an embodiment transmits a transmission wave, such as a millimeter-wave radar wave, to a subject 200 and measures (estimates) the heart rate of the subject 200 based on the result of receiving a reflected wave reflected from the chest where the heart of the subject 200 is located. As described above, the subject 200 may be a human or an animal. In this case, for example, a component assumed to be the envelope of the heart rate can be extracted by frequency filtering vibrations at the location of the subject 200 detected by radar. Once the component assumed to be the envelope of the heart rate is extracted, the interval between peaks of the envelope can be taken as the heart rate interval, thereby calculating an approximate heart rate interval. Here, an approximation can be used in which the peak of the heart rate envelope roughly coincides with the R peak of an electrocardiogram. For this reason, the "heart rate interval" is also referred to as the RR interval or RRI (RR interval), similar to the term used in electrocardiograms.

[0085] Here, we will consider a method for estimating the heartbeat interval of subject 200 from the results of the above-mentioned 2D-FFT, CFAR processing, and direction-of-arrival estimation. First, we will explain how a person's heartbeat or body movement appears as a result of the 2D-FFT performed in Figs. 4 and 5.

[0086] FIG. 9 is a diagram showing an example of the results of receiving a reflected wave of a transmitted wave transmitted to the subject 200 and performing 2D-FFT processing. FIG. 9 shows a spectrum indicating the heart sounds and body movement of the subject 200 as a result of the 2D-FFT. In FIG. 9, the horizontal axis represents distance (range) and the vertical axis represents velocity. The signal processing unit 10 (e.g., the heartbeat extraction unit 13) of the electronic device 1 according to an embodiment may extract, for example, a peak Hm as shown in FIG. 9 as body movement such as the heartbeat of the subject 200. Here, the spectral components indicated by the peak Hm in FIG. 9 include not only the heart sounds and the envelope of the heart sounds of the subject 200 but also body movement. Extracting the heartbeat interval requires extracting body movement, etc., so frequency filtering, for example, may be performed. The frequency filtering performed here may be, for example, a band-pass filter, a high-pass filter, and / or a low-pass filter targeting a frequency range of 0.5 Hz to 10 Hz.

[0087] FIG. 10 is a diagram illustrating a method for detecting peaks based on the envelope waveform of heart sounds obtained by the above-described frequency filtering. The graph in FIG. 10 shows an example of the time change of the envelope waveform of heart sounds extracted by the above-described frequency filtering. As shown, the envelope waveform in FIG. 10 contains many peaks. Meanwhile, it is known that the heartbeat of subject 200 falls within a range of approximately 50 to 130 beats per minute. Therefore, by selecting, from the many peaks shown in FIG. 10, peaks with a time interval between 0.4 and 0.8 seconds, which is the reciprocal of the number of heartbeats per minute, the approximate heartbeat interval of subject 200 can be calculated. For example, the peak indicated by the downward arrow in FIG. 10 may be selected as the approximate heartbeat interval of subject 200.

[0088] 11 is a flowchart showing an example of the above-mentioned operation of estimating a heartbeat interval. The above-mentioned operation of estimating a heartbeat interval will be outlined below with reference to FIG.

[0089] Fig. 11 shows the operation of the electronic device 1 according to one embodiment after receiving a reflected wave. That is, as a premise for the operation shown in Fig. 11, the electronic device 1 shown in Fig. 2 transmits a transmission wave (transmission signal) from the transmitting antenna array 24. Then, at least a part of the transmission wave transmitted from the electronic device 1 is reflected by the subject 200 (for example, the chest) and becomes a reflected wave. Then, the electronic device 1 shown in Fig. 2 receives such a reflected wave from the receiving antenna array 31. Then, the operation shown in Fig. 11 starts.

[0090] 11 starts, first, in step S110, the signal processing unit 10 of the electronic device 1 processes the received signal (received signal). The signal processing performed in step S110 may include, for example, the above-mentioned 2D-FFT, CFAR processing, and / or direction-of-arrival estimation. Such operations may be performed by, for example, the received signal processing unit 12 of the signal processing unit 10.

[0091] Next, in step S120, the signal processing unit 10 extracts a vibration source from the information processed in step S110. The signal processing performed in step S120 may include, for example, filtering the data resulting from 2D-FFT processing. In step S120, the signal processing unit 10 may extract spectral components only at the position where the subject 200 is present. Here, the position where the subject 200 is present may be identified by various known methods. This operation may be performed, for example, by the heart rate extraction unit 13 of the signal processing unit 10.

[0092] Next, in step S130, the signal processing unit 10 converts the result of the previous processing into a vibration waveform. The processing performed in step S130 may include, for example, a process of extracting phase information from the IQ data. In step S130, the signal processing unit 10 may also include a process of extracting vibration data including heart sounds from the spectral components of the 2D-FFT processing of the subject 200 extracted in step S120. This operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0093] Next, in step S140, the signal processing unit 10 extracts vibration data from the result of the previous processing. The processing performed in step S140 may include, for example, frequency filtering. In step S140, the signal processing unit 10 may extract a low-frequency signal including an envelope of the heart sounds of the subject 200 by performing frequency filtering. This operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0094] Next, in step S150, the signal processing unit 10 detects peaks of the heartbeat of the subject 200 from the results of the previous processing, and calculates the interval between the peaks to calculate the RR interval (RRI) of the subject 200. In step S150, the signal processing unit 10 may detect peaks of a low-frequency signal including an envelope of the heart sounds of the subject 200. Also, in step S150, the signal processing unit 10 may calculate and / or extract the interval between each peak. Such an operation may be performed by, for example, the calculation unit 14 of the signal processing unit 10. As described above, in step S150, the signal processing unit 10 may extract the interval between heart sounds.

[0095] Next, in step S160, the signal processing unit 10 may calculate the heart rate variability (HRV) of the subject 200. In step S160, the signal processing unit 10 may calculate the HRV of the subject 200 by calculating the spectral density of the time-series data of the RRI. When calculating the power spectral density of the time-series data of the RRI, a frequency analysis of the time-series waveform of the RRI may be performed, for example, by using the Welch method. This operation may be performed by, for example, the calculation unit 14 of the signal processing unit 10. Furthermore, in step S160, the signal processing unit 10 may analyze the spectrum from the processing result of the previous stage.

[0096] As described above, the electronic device 1 according to an embodiment extracts components that are thought to be the envelope of the heartbeat by frequency filtering the vibrations at the position where the subject 200 is present, and determines the interval between the peaks of the envelope as the heartbeat interval. In this way, the electronic device 1 according to an embodiment can calculate the approximate heartbeat interval of the subject 200.

[0097] In the calculation of the heartbeat interval as described above, for example, as shown in FIG. 10, several peaks are present within a time span of several tens of milliseconds. Therefore, there is a certain degree of uncertainty in selecting the heartbeat peaks. As a result, the accuracy of the calculated heartbeat interval also has an error of several tens of milliseconds. For example, in the calculation of the heartbeat interval as described above, an error of at least 20 milliseconds occurs when compared with the instantaneous RRI obtained by an electrocardiograph. With this accuracy, it is difficult to calculate heart rate variability (HRV) and perform analysis of a person's autonomic nervous system and / or emotions. In other words, in the calculation of the heartbeat interval as described above, it is difficult to perform more advanced medical analysis by extracting heart sounds.

[0098] Therefore, it is conceivable to further cut the peaks on the high frequency side by using frequency filtering so that the time interval between heartbeats becomes approximately 0.4 to 0.8 seconds. However, since this does not exceed the accuracy of the signal information shown in Figure 10, it seems difficult to reduce the error of approximately several tens of milliseconds.

[0099] Furthermore, the calculation of the heartbeat intervals described above does not involve extracting heart sounds, and therefore it is difficult to obtain information that contributes to diagnosis (for example, auscultation during medical treatment) based on the acoustic properties of the heart sounds themselves.

[0100] Therefore, the electronic device 1 according to one embodiment further improves on the above-described method. As a result, the electronic device 1 according to one embodiment analyzes the heart sounds themselves by extracting the heart sounds using a radar that uses a high frequency band of, for example, millimeter waves or higher, and extracts accurate heartbeat intervals. Such a method will be described below.

[0101] In order to extract the heart sounds of the subject 200 with high accuracy, the electronic device 1 performs appropriate signal processing on the signal (chirp signal) received by the electronic device 1 in an appropriate order to narrow down the subspace and subspace basis in which the signal components of the heart sounds of the subject 200 exist. The electronic device 1 according to an embodiment may employ different basis vectors in the linear space in which the heart sound signals of the subject 200 exist, describe the space using an appropriate coordinate system, and extract the subspace based on the respective coordinates. Through such processing, the electronic device 1 according to an embodiment can search for the subspace in which the heart sounds of the subject 200 exist. In an embodiment, an appropriate coordinate system may be used depending on the purpose. For example, a coordinate system of a space obtained by 2D-FFT processing, a coordinate system of a time-series signal obtained by temporally contracting a chirp signal, a coordinate system based on the Fourier transform of the time-series signal, or a coordinate system based on continuous / discrete wavelets may be used.

[0102] The electronic device 1 according to one embodiment may perform the following characteristic processing on the received signal in a step-by-step procedure. The characteristic processing by the electronic device 1 according to one embodiment is outlined below. The electronic device 1 according to one embodiment performs three characteristic processing steps. That is, the electronic device 1 according to one embodiment performs (1) extraction of a subspace, (2) generation of an envelope and selection of a best heart sound, and (3) statistical signal processing for calculating the heartbeat interval. Each processing step will be described in more detail below.

[0103] (1) Extraction of subspace First, the electronic device 1 according to an embodiment executes a process for removing data that is considered unnecessary in a linear space from a received signal. The electronic device 1 according to an embodiment may execute a process for removing unnecessary subspaces to leave necessary subspaces. Specifically, the electronic device 1 may execute the following process.

[0104] (1-1) First step: Dimension reduction In the first stage, an appropriate window function is applied to the 2D-FFT processing of the chirp signal received by the electronic device 1, and only the micro-Doppler components of the point cloud where the subject 200 (e.g., a person or animal) is present are extracted by estimating the direction of arrival. Here, multiple window functions are generated around the approximate position of the target, with multiple distance ranges at their centers, and these multiple window functions are applied. Hereinafter, this process of applying multiple window functions is referred to as "multi-window processing." Here, one vibration time series signal may be generated for one window function, i.e., one center distance range.

[0105] (1-2) Second stage: Reducing the dimension of the subspace In the second stage, principal component analysis and / or singular value decomposition of the set of time-series waveforms of the vibrations extracted in the processing of the first stage is performed on the micro-Doppler signals.

[0106] (1-3) Third step: Reducing the dimension of the subspace In the third stage, frequency filtering is performed using a short-time Fourier transform, a continuous wavelet transform, and / or a band-pass filter.

[0107] (1-4) Fourth stage: Reducing the dimension of the subspace In the fourth step, heart sounds are extracted by performing multi-resolution analysis using discrete wavelet transforms with wavelet and scaling functions appropriate for the heart sounds.

[0108] (2) Envelope generation and selection of the best heart sound Next, to facilitate processing of the extracted heart sound data, the electronic device 1 according to an embodiment may execute a process to generate an envelope from the waveform of the extracted heart sound. Furthermore, the electronic device 1 according to an embodiment may execute a process to select (extract) the best heart sound data from multiple pieces of heart sound data. Specifically, the electronic device 1 may execute the following process.

[0109] (2-1) Envelope generation Here, an envelope is generated from the heart sounds extracted in the above (1) subspace extraction. The envelope generation process may use any of the following processes, for example, continuous wavelet transform, discrete wavelet transform, wavelet scattering coefficients, Mel-frequency cepstrum coefficients, or moving variance.

[0110] (2-2) Selection of the best heart sound Next, the best heart sound is selected from the generated envelope. The number of heart sounds generated by the subspace extraction in (1) above is equal to the number of window functions. Therefore, here, the optimal heart sound is extracted (selected) from the number of heart sounds that are equal to the number of window functions. First, a pre-trained learning classifier may be used to calculate scores for the heart sounds. In this case, the learning classifier may be, for example, an autoencoder, a long short-term memory (LSTM), another neural network, a support vector machine, or a decision tree. The best heart sound is then determined based on the calculated scores. At the same time, associated calculation results may be obtained from the learning classifier. The learning classifier may be any of a linear support vector machine, a simple perceptron, a classifier using logistic regression, a decision tree (classification tree), a k-nearest neighbor algorithm, a random forest, a nonlinear support vector machine, a neural network, or a combination of these.

[0111] (3) Statistical signal processing for calculating heartbeat intervals Here, the heart sound intervals are calculated by performing statistical processing on the generated envelope of the best heart sound. In most cases, including healthy individuals, heart sounds include a first heart sound (S1) and a second heart sound (S2). When detecting the time intervals between heart sound peaks within a certain period of time, the following two intervals exist: (i) Between the first and second heart sounds (hereinafter also referred to as "between S1 and S2") (ii) Between the second and first heart sounds (hereinafter also referred to as "S2-S1 interval") The histogram or probability density distribution of these intervals is a mixed Gaussian distribution or a single Gaussian distribution. Therefore, accurate heart sound intervals can be calculated based on this distribution information using, for example, simple dynamic programming (DP) matching.

[0112] According to the electronic device 1 of one embodiment, the envelope waveform of the heart sound is obtained through the above-mentioned processing steps, and the positive and negative peaks are detected. The interval between the peaks is then appropriately calculated, thereby enabling the RR interval (RRI / RR interval) to be calculated with high accuracy.

[0113] Next, the operation of the electronic device 1 according to the embodiment will be described in more detail.

[0114] Fig. 12 is a flowchart showing an example of the operation performed by the electronic device 1 according to an embodiment. Fig. 13 is a flowchart showing in more detail an example of the operation of step S16 in Fig. 12. Hereinafter, the flow of the operation performed by the electronic device 1 according to an embodiment will be described with reference to Figs. 12 and 13.

[0115] Step S11 shown in Fig. 12 can be performed in the same manner as the operation in step S110 shown in Fig. 11. That is, when the operation shown in Fig. 12 starts, first, in step S11, the signal processing unit 10 of the electronic device 1 processes a received signal (received signal). The signal processing performed in step S11 may include, for example, the above-mentioned 2D-FFT, CFAR processing, and / or direction-of-arrival estimation. Such an operation may be performed by, for example, the received signal processing unit 12 of the signal processing unit 10.

[0116] In step S12, the signal processing unit 10 performs multi-window processing. In step S12, the signal processing unit 10 may perform a process of determining candidates for human or animal clusters and / or a filtering process (multi-window processing) on ​​data that has been subjected to 2D-FFT processing using multiple window functions.

[0117] In step S12, the signal processing unit 10 may perform multi-window processing to generate multiple heart sound candidates as a preliminary step to selecting the best heart sound candidate. Here, the signal processing unit 10 may perform processing on the 2D-FFT-processed data to group point clouds corresponding to people or animals into clusters. The signal processing unit 10 may also perform multi-window processing by applying multiple window functions to the selected clusters. The multi-window processing performed here may include processing to generate 2D-FFT-processed data that serves as the basis for the multiple heart sound candidates.

[0118] In step S12, the signal processing unit 10 may generate multiple candidate heart sounds by extracting only the region where the subject 200 is present on the range-Doppler plane calculated by 2D-FFT using an appropriate window function. In step S12, the signal processing unit 10 may provide multiple window functions as the appropriate window function. Here, the signal processing unit 10 may use window functions such as a Hanning window, a Hamming window, and a Blackman-Harris window, for example.

[0119] In step S12, the signal processing unit 10 may detect a vibration source (target) including heart sounds and / or body movements including breathing of the subject 200, based on, for example, the following first to third procedures.

[0120] (Step 1) The signal processing unit 10 classifies the points on the range-Doppler plane that exceed the CFAR threshold into areas of a predetermined angle based on the result of direction-of-arrival estimation (see FIGS. 5 and 6). For example, if the angle on the xy plane shown in FIG. 6 is θ, the points may be classified into areas A to C of the following angles. Area A: -10deg.<θ<10deg. Area B: -20deg.<θ≦-10deg. Area C: 10deg.≦θ<20deg.

[0121] (Second step) The signal processing unit 10 applies clustering to a group of points that exceed the CFAR threshold on a range-Doppler plane such as S1 in Fig. 5, within a group of areas of a certain angle classified in the first step. Here, a method such as DBSCAN may be applied as the clustering method.

[0122] (Third step) Assuming that L clusters have been processed in the second step, the signal processing unit 10 calculates the deviation D of the number of bins in the Doppler direction for the l-th cluster. dev [l] is a threshold D dev,th As a result of this comparison, the signal processing unit 10 determines whether dev [l]≧D dev,th Only those who satisfy the condition are determined to be the subject 200 and a flag HF[l] is set. That is, the signal processing unit 10 may perform the process shown in the following [Pseudo Code 1], for example.

[0123] [Pseudocode 1] for l = 1 to L do if D dev [l] ≧ D dev,th HF[l] = 1 else HF[l] = 0 end if end do

[0124] In step S13, the signal processing unit 10 restores the set of chirp signals. In step S13, the signal processing unit 10 may restore the set of chirp signals by performing an inverse discrete Fourier transform (2D-IFFT (inverse fast Fourier transform)) on the plurality of generated 2D-FFT data.

[0125] Fig. 14 is a diagram illustrating multi-window processing by the electronic device 1 according to one embodiment. Fig. 14 illustrates multi-window processing for generating multiple time-series signals from a point cloud belonging to a cluster selected by the above-described pseudocode 1 using window functions shifted in multiple range directions.

[0126] In the upper part of FIG. 14, the horizontal axis indicates distance (Range) and the vertical axis indicates Doppler velocity. The upper part of FIG. 14 shows the results of clustering performed on a point cloud of 2D-FFT data at a certain time. The upper part of FIG. 14 shows how three clusters, Cluster 1, Cluster 2, and Cluster 3, are generated as a result of the clustering. The middle part of FIG. 14 conceptually shows how the signal processing unit 10 performs multi-window processing on each of Cluster 1, Cluster 2, and Cluster 3. The middle part of FIG. 14 shows how the signal processing unit 10 performs multi-window processing on each of Cluster 1, Cluster 2, and Cluster 3 by gradually shifting the window function in the distance (Range) direction. The lower part of FIG. 14 shows the results of calculating vibration velocities by the signal processing unit 10 performing 2D-IFFT on each of Cluster 1, Cluster 2, and Cluster 3. The lower part of FIG. 14 shows how time-series vibration signals are generated by performing 2D-IFFT on each of the clusters. In practice, a window function is applied to each cluster shown in FIG. 14 multiple times while being shifted slightly in the range direction, thereby obtaining multiple time-series vibration signals.

[0127] In step S13 of Fig. 11, the signal processing unit 10 applies multiple window functions to a cluster formed from a group of points for which flag HF[l] = 1 has been selected in accordance with the above-mentioned [Pseudo Code 1]. This process generates 2D-FFT data corresponding to each window function. Here, the 2D-FFT data generated by the l-th window function is expressed as in the following equation (1).

number

[0128] In this case, in step S13, the signal processing unit 10 can calculate the time-series signal waveform of the vibration velocity according to the following equation (2): The signal waveform is restored to a set of chirp signals by a two-dimensional inverse fast Fourier transform (2D-IFFT).

number

[0129] Below, as an example, only one cluster, for example, cluster 1 shown in FIG. 14, will be described.

[0130] In step S14, the signal processing unit 10 analyzes the principal components of the signal and / or removes noise. Step S14 may be singular value decomposition (SVD) for pre-processing the signal noise. This process may be performed to remove low-energy noise and / or low-level noise that has been introduced due to the uncertainty of the Fourier transform.

[0131] In step S14, the signal processing unit 10 calculates a set S of received chirp signals extracted in the 2D-FFT plane. l Let U denote the matrix of left singular vectors, Σ denote the matrix of diagonal singular values, and V denote the matrix of right singular vectors. Then, the set of received chirp signals S l By singular value decomposition of the above, the following equation (3) is obtained.

number

[0132] Next, the signal processing unit 10 limits the number of row vectors in the row of the left singular vectors in the above formula (3) (U ext ) The signal processing unit 10 may also limit the number of diagonal elements of the diagonal matrix of the singular values ​​of the above equation (3) (Σ ext ) can be used. This eliminates unwanted noise signals and vibration components other than the desired position. As a result, S l The signal S projected onto the subspace of the target signal l ext is expressed as the following equation (4).

number

[0133] FIG. 15 is a diagram illustrating the relationship between the ranks representing the target signal and the noise signal in singular value decomposition. FIG. 15 is a diagram in which the results of SVD are arranged in descending order of singular value (corresponding to energy), with the horizontal axis representing the rank and the vertical axis representing the singular value. The present disclosure adopts the idea that a signal extracted that is equal to or greater than a certain singular value becomes the target signal. In this case, the rank of the singular value that determines whether or not it is the target signal is the rank region of 30 or less, indicated by the dark gray area in the graph of FIG. 15. As shown in FIG. 15, the region of 30 or less is the region of the target signal. In FIG. 15, the rank portion of this target signal is 1 or more and 30 or less. Therefore, in the present disclosure, for example, singular vectors corresponding to left and right singular values ​​in the rank region of 1 or more and 30 or less may be extracted.

[0134] In the above example, the rank portion of the target signal is set to 1 or more and 30 or less. The maximum rank may basically be determined empirically. The maximum rank (rank 30 in this disclosure) may also be determined using a statistical method. In the graph of FIG. 15, the singular value suddenly decreases at rank 130, and signals at ranks after that are basically noise. Therefore, signals at ranks after that may be unnecessary.

[0135] In the graph of Fig. 15, the left and right singular vectors (vectors spanning the signal space) corresponding to the singular values ​​in the area of ​​rank 31 to 130 also contain some target signal components. These left and right singular vectors are assumed to be mainly caused by unnecessary minute vibrations and / or artificial noise (artifacts) generated by radar signal processing. Therefore, such elements may be discarded.

[0136] After noise is removed by SVD in step S14, the signal processing unit 10 converts the noise-removed result into a signal waveform (step S15). In step S15, the signal processing unit 10 may convert the beat signal (IQ data) into a time-series signal of vibration.

[0137] In step S15, the signal processing unit 10 obtains the vector shown in the following equation (5) by taking the sum of the column vectors of the chirp signals based on the above equation (4).

number

[0138] Finally, in step S15, s l ext,sum By taking the argument and derivative on the Gaussian plane, the vector representing the vibration velocity is calculated as shown in the following equation (6).

number

[0139] Next, in step S16, the signal processing unit 10 extracts heart sounds and / or analyzes heartbeat intervals from the obtained vibration waveforms. Specifically, in step S16, the signal processing unit 10 extracts heart sounds from the vibration waveform shown in the above equation (6). In step S16, the signal processing unit 10 may execute a process to generate a heart sound waveform, a process to calculate the heartbeat interval (RRI), and / or a process to calculate the heart rate variability (HRV). This enables the signal processing unit 10 to calculate the heartbeat interval. This process may be executed on waveforms of multiple vibration velocities corresponding to each window function.

[0140] As described above, the process of step S16 shown in Fig. 12 may, in more detail, include at least a part of the processes of steps S21 to S27 shown in Fig. 13. Each of steps S21 to S27 shown in Fig. 13 will be described in more detail below.

[0141] 13, the signal processing unit 10 performs a process of removing noise from a time-series waveform (denoising). Here, the process of removing noise may use an Empirical Bayes method or a wavelet method such as a Continuous Wavelet Transform (CWT).

[0142] More specifically, in step S21, the signal processing unit 10 calculates the vibration velocity vector v corresponding to the l-th window function. l vib Further preprocessing is performed to remove unnecessary noise.

[0143] In step S21, the signal processing unit 10 may perform noise removal processing by limiting the band using an empirical Bayes method and / or a continuous wavelet. Also, in step S21, the signal processing unit 10 may perform noise removal processing by frequency subtraction using a noise profile for artifact noise and the like accompanying the nonlinear processing from step S11 to step S14 shown in Fig. 12.

[0144] Next, in step S22, the signal processing unit 10 extracts the waveform of the target signal, i.e., the waveform of the heart sound. In this case, for example, a discrete wavelet technique such as maximum overlap discrete wavelet transform (MODWT) may be used.

[0145] More specifically, in step S22, the signal processing unit 10 may perform multiresolution analysis using a discrete wavelet transform on the denoised signal as preprocessing, employing a wavelet waveform having a waveform similar to the heart sound waveform. In this way, in step S22, the signal processing unit 10 empirically extracts only the subspace of the level of multiresolution analysis in which the heart sounds of the subject 200 exist. In this way, in step S22, the signal processing unit 10 may extract the heart sounds of the subject 200. Specifically, the signal processing unit 10 may use a maximum overlap multiresolution analysis (MODWT) or the like to improve the temporal resolution. Furthermore, wavelet bases suitable for extracting heartbeats may be, for example, Symlet and Daubechies. Furthermore, the order of these may be set appropriately each time.

[0146] The denoising process in step S21 is performed on the signal velocity vector v l vib The signal sent to v l vib,dn Furthermore, the real part Re(v l vib,dn) may be subjected to multi-resolution analysis as shown in Fig. 16. Fig. 16 is a diagram conceptually showing multi-resolution analysis using discrete wavelet transform. Then, the signal processing unit 10 obtains the cardiac sound waveform h by reconstructing the waveform by limiting the empirically appropriate level to j0∈N in the following equation (7).

number

[0147] In this way, the electronic device 1 according to one embodiment can obtain a sound waveform h such as the waveforms shown in Figures 26 and 27, which will be described later. This result can be directly used in medical examinations of heart sounds, etc.

[0148] Next, in step S23, the signal processing unit 10 generates an envelope waveform. In step S23, the signal processing unit 10 extracts an envelope waveform by generating a scalogram using a continuous wavelet transform and converting it into a one-dimensional scalogram in order to detect (extract) the energy peak of the target signal (cardiac sound waveform) extracted in step S22. Here, discrete / continuous wavelet transform, shift variance, and / or Hilbert transform may be used.

[0149] More specifically, in step S23, the signal processing unit 10 calculates the cardiac waveform h l To obtain the envelope waveform of , a scalogram may be obtained by continuous wavelet transform. The signal processing unit 10 calculates the sum of the scalogram over a certain range of the frequency axis from f1 to f2 at each time. As a result, the signal processing unit 10 obtains the one-dimensional waveform s as shown in Figs. 17 and 18. l h17 and 18 show examples of the analysis results of the continuous wavelet transform. This analysis can be replaced with the discrete wavelet transform, with sufficient attention paid to the resolution.

[0150] Figure 17 shows the heart sound waveform h l Fig. 17 is a diagram showing the time change of the normalized vibration frequency. The horizontal axis of Fig. 17 represents time in units of the number of samples, and the vertical axis of Fig. 17 represents the normalized vibration frequency for each time.

[0151] FIG. 18 shows the heart sound waveform h shown in FIG. l Fig. 18 is a diagram showing the result of applying a continuous wavelet transform to one-dimensionalize the scalogram of Fig. 18. Fig. 18 is a diagram showing the change in vibration velocity over time. The horizontal axis of Fig. 18 represents time in units of the number of samples, and the vertical axis of Fig. 18 represents vibration velocity.

[0152] Here, the matrix representing the absolute value of the scalogram satisfies the following equation (8).

number

[0153] [Pseudocode 2] for m=1:M

number

[0154] Next, in step S24, the signal processing unit 10 selects the best heart sound. In step S24, the signal processing unit 10 may perform a machine learning classification process to extract the best heart sound envelope from the envelope waveforms of the multiple heart sounds extracted in step S23. Here, machine learning such as an autoencoder, a long short term memory (LSTM), or other neural network, or a support vector machine (SVM) may be used.

[0155] More specifically, in step S24, the signal processing unit 10 calculates a vector e l h Here, the signal processing unit 10 selects the optimum e l h To select the optimal image, each cost function may be scored and the one with the smallest cost function may be selected. Here, the cost function is calculated using the reconstruction error by an autoencoder.

[0156] Figure 19 and Figures 20(A) and 20(B) are diagrams conceptually explaining the process of classifying and identifying the best heart sound envelope. Figure 19 is a diagram conceptually showing how an autoencoder learns a large amount of training data. Figures 20(A) and 20(B) are diagrams showing an example of verifying errors when data is reconstructed by an autoencoder. The vertical axes of Figures 20(A) and 20(B) represent arbitrary units determined by the digital scale of signal processing.

[0157] In step S24, the signal processing unit 10 may perform a process of identifying the envelope of the heart sounds to select the best heart sound, as in the following sub-steps 241 to 243.

[0158] <Substep S241> In sub-step S241, the signal processing unit 10 may prepare a large amount of training data for a data set corresponding to the envelopes of the first and second sounds among the heart sound envelopes shown in the upper part of Fig. 19, and train the autoencoder using the data. Here, the number of vector dimensions of each piece of training data is denoted as J. The upper part of Fig. 19 shows data augmentation and automatic extraction of one phoneme of an appropriate heart sound envelope. By training the training data in this way, a trained autoencoder is generated, as shown in the lower part of Fig. 19.

[0159] <Substep S242> In sub-step S242, the signal processing unit 10 may perform reconstruction of the trained autoencoder as shown in Figures 20(A) and 20(B). In this case, the signal processing unit 10 performs reconstruction of the trained autoencoder by using the envelope vector e l h The i-th vector e is cut sequentially by the length of J. l,i h Next, the signal processor 10 performs reconstruction by converting the original envelope waveform vector e l,i h and the mean absolute percentage error (MAPE) between the reconstructed envelope waveform vector shown in the following equation (10) is calculated.

number

[0160] In this case, the signal processing unit 10 may calculate the MAPE while sequentially shifting the window function for calculating the cost function, as shown in the following [Pseudo Code 3].

[0161] [Pseudocode 3]

number

number

[0162] where length is a function that represents the length of the signal vector, and trunc is a function that rounds by truncating the decimal point. l h The final part of the length less than J is truncated and not included in the calculation.

[0163] In Figures 20(A) and 20(B), one of the solid and dashed line graphs may represent a waveform based on detected data, and the other of the solid and dashed line graphs may represent a waveform based on reconstructed data. Figure 20(A) represents data whose MAPE exceeds a threshold, i.e., data that cannot be well predicted (cannot be properly reconstructed), and thus shows a waveform that cannot be considered relatively good. On the other hand, Figure 20(B) represents data whose MAPE does not exceed a threshold, i.e., data that can be well predicted (can be properly reconstructed), and thus shows a waveform that can be considered relatively good.

[0164] <Substep S243> 21(A) and 21(B) are diagrams showing examples of MAPE. 21(A) and 21(B) are diagrams explaining the process of extracting the minimum point of MAPE from the results of reconstruction by the autoencoder in order to select the best heart sound. The horizontal axis of 21(A) and 21(B) represents time in units of number of samples, and the vertical axis of 21(A) and 21(B) represents MAPE (unit: %). 21(A) shows the MAPE of the best heart sound envelope. l env FIG. 21(B) shows an example of the MAPE of a non-optimal heart sound envelope. l env Here is an example:

[0165] In the electronic device 1 according to one embodiment, the signal processing unit 10 calculates a MAPE value based on a certain threshold MAPE in the MAPE shown in FIG. 21(A) or FIG. 21(B). th Then, the signal processing unit 10 sets the threshold MAPE thFor example, the dashed lines shown below the graphs in each of Figures 21(A) and 21(B) represent the MAPE th = 10. Here, the cost function is the number of minimum points below the threshold. Such a cost function can be expressed mathematically as the following equation (13).

number

[0166] In the above formula (13), LocalMin is a function that lists the local minimum points. In the electronic device 1 according to one embodiment, the signal processor 10 may select the best heart sound that minimizes the cost function shown in the above formula (13). That is, the signal processor 10 selects the best heart sound number l best may be selected as shown in the following equation (14):

number

[0167] The MAPE shown in Figure 21(A) l env is the threshold MAPE th There are a relatively large number of minimum points below the MAPE (=10%). Therefore, the signal processing unit 10 may use the envelope shown in FIG. 21(A) as an example of the best envelope of the heart sound. Also, the MAPE shown in FIG. 21(B) l env is the threshold MAPE th 21B is an example of a suboptimal envelope of a heart sound.

[0168] Next, in step S25, the signal processing unit 10 statistically analyzes the best heart sound. In step S25, the signal processing unit 10 may extract positive and negative peaks from the envelope of the best heart sound selected in step S24 and perform statistical analysis between the peaks. This generates information for calculating the RRI. In step S25, the signal processing unit 10 may use a Gaussian Mixture Model (GMM) distribution estimation, an EM algorithm (expectation-maximization algorithm), a variational Bayesian method, or the like.

[0169] The process of generating a frequency distribution (histogram) of the peak intervals of the heart sound envelope will be described below with reference to Figures 22 and 23. Figure 22 is an enlarged view of an example of the envelope of two heart sounds. The horizontal axis of Figure 22 represents time in units of the number of samples, and the vertical axis of Figure 22 represents the signal level. Figure 23 is a diagram showing an example of a histogram of peak intervals. The horizontal axis of Figure 23 represents the time between peaks, and the vertical axis of Figure 23 represents the frequency.

[0170] FIG. 22 shows an example of the envelope of the first heart sound (S1) and the second heart sound (S2). Also shown in FIG. 22 are the time intervals between S1 and S2 (S1-S2 interval) and between S2 and S1 (S2-S1 interval). FIG. 23 shows the frequency distributions between S1 and S2 and between S2 and S1. FIG. 23 shows histograms corresponding to both the S1-S2 interval and the S2-S1 interval.

[0171] The probability density distribution obtained by normalizing this histogram is a Gaussian mixture distribution of two clusters. Therefore, in order to later estimate the RRI based on this information, it is necessary to estimate the Gaussian mixture distribution. Here, the Gaussian mixture distribution is a distribution described by the linear sum of multiple Gaussian distributions, as shown in the following equations (15) and (16). As shown in the histogram of peak intervals in Figure 23, the probability distribution between heart sound peaks is this Gaussian mixture distribution.

number

number

[0172] The following describes a process for estimating a Gaussian mixture distribution using the variational Bayes method. This process may include the following two processes, [First Process] and [Second Process]. The [First Process] and [Second Process] will be described in more detail below.

[0173] [First process] The envelope of the heart sound (shown in the following equation (17)) is cut out for each time frame numbered k=1, 2, . . . , K for each length T.

number

[0174] In the above [first process], the signal processing unit 10 divides the heart sound selected as the best heart sound (shown in the above formula (17)) into K time frames. That is, the signal processing unit 10 generates an extracted envelope waveform as shown in the following formula (18).

number

[0175] FIG. 24 is a diagram for explaining the process of extracting the envelope of a heart sound. In the envelope of a heart sound shown in the upper part of FIG. 24, the time windows for extracting for time frames t=1 and t=2 are shown. In the envelope of a heart sound shown in the lower part of FIG. 24, the time windows for extracting for time frames t=T-1 and t=T are shown. Here, T and t are discrete times and represent non-negative integers. In addition, in the envelope of a heart sound shown in the upper part of FIG. 24, the width of the time window w win and the overlap width w o Here, the time window movement step is w step =w win -w o This becomes:

[0176] [Second process] The mixed Gaussian distribution latent variable at time t=T is input as the prior distribution, and at t=T+1, the latent variable is updated using the variational Bayes method.

[0177] In the above [second process], the signal processing unit 10 performs Bayesian estimation for the time frame t=k using the variational Bayes method described below, including the latent parameters of the Gaussian mixture distribution.

[0178] Here, we will outline the estimation of variational Bayes. Variational Bayes is a method for estimating the posterior distribution of a latent variable Z. Roughly speaking, variational Bayes is a method for approximating an approximate posterior distribution q(Z) to the actual posterior distribution p(Z|X) using variational calculus. In variational Bayes, latent variables and variables are synonymous. In variational Bayes, all variables are treated as latent variables whose true values ​​cannot be calculated, i.e., as random variables. Below, we will explain an overview of variational Bayes by generalizing latent variables.

[0179] In the variational Bayesian method, instead of minimizing the Kullback-Leibler divergence (KL divergence) written as in the following equation (19), an approximation is performed by maximizing the following equation (20), which is called the Evidence of Lower Bound (ELBO).

number

number

[0180] That is, the signal processing unit 10 can obtain the approximate posterior distribution q(Z) of the latent variable Z by solving the following equation (21).

number

number

[0181] The signal processing unit 10 may perform processing to solve the Euler-Lagrange equation shown in the following equation (23) for the above equation (19) under the condition of the above equation (22).

number

number

number

[0182] Below, we will explain a method for utilizing the variational Bayes method described above to estimate an actual Gaussian mixture distribution.

[0183] First, it is necessary to group the latent variables of the approximate posterior distribution and perform mean field approximation. In order to use the variational Bayes update formula (24), first, for the latent variables of the mixed Gaussian distribution, taking into account their respective dependencies, the joint probability density distribution is decomposed into each probability distribution using Bayes' theorem as shown in the following formula (26).

number

[0184] The conjugate prior distribution of the probability distribution that is each factor in the above formula (26) is set as shown in the following formulas (27) to (31). When the prior distribution and the posterior distribution for a certain likelihood function have the same functional form, the prior distribution and the posterior distribution are called conjugate prior distributions for the likelihood function.

number

number

number

number

number

[0185] Here, in terms of equation (29), P(π|α) is defined as α=[α1,α2,…,α K ] is a Dirichlet distribution with parameters. Also, B(α) is a β function expanded to multivariate. Regarding equation (31), W(Σ k |W,ν) is a Σ k where ν is the number of data vectors. D is the number of variates. W is the distribution of Σ k C(W,ν) is a constant calculated using the Γ function as shown in the following equation (32).

number

[0186] Next, the approximate posterior distribution is grouped into one-hot vector z and other variables, and Equation (24) is specifically defined as the following Equation (33).

number

[0187] Unlike the EM algorithm, the variational Bayes method does not distinguish between latent variables and general variables. Therefore, there is no particular meaning in distinguishing between the E-step (Expectation step) and the M-step (Maximization step) in the variational Bayes method. However, for convenience, we will define the above two steps as follows: E-step: Take the expectation for group 2 and update the approximate posterior distribution q(z) of z. M-step: Take the expectation of q(z) for group 1 obtained in the E-step and update the approximate posterior distribution q(π,μ,Σ) for group 2.

[0188] Hereinafter, as signal processing performed by the signal processing unit 10 in the electronic device 1 according to one embodiment, a variational Bayesian method process for a Gaussian mixture distribution from time t=T−1 to time t=T will be described. This process may include the following five substeps, substep 251 to substep 255, sandwiching the above-mentioned E-step and M-step therebetween.

[0189] <Substep S251> In sub-step S251, each latent variable is initialized (when T = 0), or the latent variable at t = T-1 is inherited as a prior distribution (T ≥ 1). Here, the latent variable of the posterior distribution calculated at time t = T is given as the initial value at time t = T. Here, the latent variable of the posterior distribution calculated at time t = T is α| t=T , β| t=T-1 ,ν| t=T-1 , m| t=T-1 ,W| t=T-1 Also, the initial value at time t = T is α0| t=T , β0| t=T , ν0| t=T , m0| t=T , W0|t=T The subscript 0 is used to indicate the initial value.

[0190] <Substep S252> In substep S251, the approximate posterior distribution q(z) of z is calculated by the E-step, and the result is used to calculate the burden rate r nk (the posterior probability of one-hot vector) is calculated. Here, as the E-step, the above formula (33) is used to rewrite the above formula (24) as the following formula (34).

number

[0191] Using the above formula (34), the burden rate r is calculated using the following formulas (35) and (39). nk Calculate (posterior probability of one-hot vector).

number

number

number

number

number

[0192] <Substep S253> In sub-step S253, as an M-step, the burden rate r nk The latent variables (group 2) of the Gaussian mixture distribution are updated using the above formula (24). Here, n is the index of the data. Also, k is the index of the Gaussian distribution. Here, as an M-step, the above formula (24) is rewritten as the following formula (40).

number

[0193] The parameters are updated using the above equation (40) and the following equations (41) to (48).

number

number

number

number

number

number

number

number

[0194] <Substep S254> In sub-step S254, convergence is determined by repeating sub-step S252 and sub-step S253, and the processing of sub-step S252 and sub-step S253 is terminated. Specifically, convergence may be determined when the increment of the log-likelihood function or the increment of the ELBO between iteration steps becomes equal to or less than a set value, or when the maximum number of iterations is exceeded.

[0195] In sub-step S254, the ELBO shown in equation (20) can be used as an index for determining convergence. When equation (20) is rewritten in accordance with the estimation of a Gaussian mixture distribution, it becomes the following equation (49).

number

number

[0196] <Substep S255> In sub-step S255, a Gaussian mixture model (GMM) is determined by taking the expected value of each distribution as a representative value for each latent variable. The variational Bayes method calculates the posterior distribution of a Gaussian mixture distribution. Therefore, in order to ultimately determine the Gaussian mixture distribution, it is necessary to determine a representative value. Here, the Gaussian mixture distribution is determined by taking the expected value for π, μ, and Σ of each latent variable. That is, the Gaussian mixture distribution is determined by using the respective expected values ​​as representative values, as shown in the following equations (51) to (53).

number

number

number

[0197] From the above, the Gaussian mixture distribution at time t=T obtained from the representative value of the approximate posterior distribution is expressed by the following equations (54) and (55). The Gaussian mixture distribution obtained in this way may be used in the next step S26.

number

number

[0198] In reality, as shown in Figures 22 and 23, in many cases, including healthy subjects, only the first heart sound (S1) and the second heart sound (S2) are prominent. In such cases, K is simplified to 2. For subjects with abnormal heartbeats, the third and / or fourth heart sounds may become apparent. In such cases, K becomes 3 or 4.

[0199] With the above, the second process is completed, and the process of step S25 may also be completed.

[0200] Next, in step S26, the signal processing unit 10 calculates the RR interval (RRI) based on the statistical information on the heart sound intervals generated in step S25.

[0201] More specifically, in step S26, the signal processing unit 10 may estimate the RRI using the Gaussian mixture distribution (the above formulas (54) and (55)) calculated in step S25. In this case, the signal processing unit 10 first estimates the distribution of the RRI using the above formulas (54) and (55).

[0202] The reproductive property of the normal distribution, that is, for two random variables X and Y that follow normal distribution, the following equation (56) holds:

number

[0203] Therefore, by considering FIG. 22 and FIG. 23 for the above equations (54) and (55), the distribution of RRI is estimated as shown in the following equation (57). However, in equation (57), μ RRI and Σ RRI is expressed as the following equation (58).

number

number

[0204] In FIG. 22 and FIG. 23, the time of the peak of the envelope of the picked-up heart sound is vectorized as t peak This can be expressed as the following equation (59).

number

[0205] Next, the mean value μ of the probability density distribution of RRI defined by the above equations (57) and (58) RRI A vector of non-negative integer multiples of t targ Then, this is expressed as the following equation (60).

number

[0206] t targ is t peak This is the target time used as a guideline for selecting candidates corresponding to points at which the RRI is obtained. Below, a DP (Dynamic Programming) matching method is performed between the above equation (59) and equation (60) to extract the peak points of the envelope of the heart sound that can be considered as the RRI.

[0207] 25 is a diagram illustrating DP matching for RRI estimation. In FIG. 25, the horizontal axis indicates the vector t peak The vertical axis is the vector t targ In Figure 25, the circles indicate peak and t targ In Figure 25, the cross marks indicate that a match was attempted using the following pseudocode 4, but no match was found. peak t targ A vector of flags indicating whether or not any element of RRI m RRI can be expressed as the following equation (61).

number

[0208] [Pseudocode 4]

number

number

number

number

[0209] Here, the above equation (62) is the variance Σ of the probability density distribution of RRI shown in the following equation (66): RRI By multiplying by a constant, t peak and t targ Distance threshold D th This means determining the

number

[0210] The above equation (63) is t peak and t targIn other words, in [Pseudocode 4], the distance between elements of t that meets the condition shown in the following equation (67) is calculated. peak This represents the process of extracting elements of the given vector as matching points.

number

[0211] As a result of the calculation in [Pseudocode 4], m RRI Let the number of non-zero elements of be M, that is, the following equation (68) holds.

number

[0212] Next, the matching flag m RRI and t peak By taking the Hadamard product (element product) of these and excluding 0, the time column vector t RRI That is, the result of extracting only the circled points in the grid pattern in FIG. 25 is expressed as the following equation (69).

number

[0213] Vector Δt showing RRI data RRI is expressed as t RRI It is calculated by taking the difference between each adjacent element of

number

[0214] Finally, the vector Δt representing the RRI data RRI In the above [Pseudo Code 4], the matching process is performed in the l-th iteration of the loop in Figure 25. t ≦Dth If not, t RRI is missing, and Δt RRI There may be outliers with very large values ​​among the elements of . This missing / outlier processing is done by using Δt RRI This can be done by allowing a certain magnification b for the average value of , and replacing it with the average value if it is greater than that.

number

[0215] [Pseudocode 5] for m=1:M-1

number

number

[0216] In this way, the RRI data that has undergone missing / outlier processing is marked with a dash and Δt ´RRI This is then filtered by a moving average filter or the like to obtain smoothed data Δt ´´RRI can be used as the final RRI data. For the cardiac sound waveforms shown in Figs. 26 and 27, the Δt obtained as described above can be used as the final RRI data. ´´RRI An example of this data is shown in Fig. 28. Figs. 26, 27, and 28 are diagrams showing examples of time-series waveforms of heart sounds and RRIs obtained by an electronic device 1 according to an embodiment. In Figs. 26 and 27, the horizontal axis represents time, and the vertical axis represents vibration velocity. In Fig. 28, the horizontal axis represents time, and the vertical axis represents RRI. Fig. 26 is a diagram showing a time-series waveform of heart sounds obtained by the electronic device 1 according to one embodiment. Fig. 27 is a diagram showing an enlarged time span of the region enclosed by the dashed line in Fig. 26.

[0217] As shown in Fig. 27, the first heart sound S1 and the second heart sound S2 can be clearly identified from the time-series waveform of heart sounds obtained by the electronic device 1 according to an embodiment. Furthermore, as shown in Fig. 27, the heart beat interval RRI calculated from the interval between the first heart sound S1 and the second heart sound S2 can also be clearly identified from the time-series waveform of heart sounds obtained by the electronic device 1 according to an embodiment. While the RRI and the heart beat interval are not strictly the same, they are considered to be approximately the same. Therefore, in the present disclosure, the RRI and the heart beat interval will be described as being the same thing.

[0218] The results shown in Fig. 28 were calculated based on the heart sounds accurately extracted in Fig. 26 and Fig. 27. Therefore, the results shown in Fig. 28 have an accuracy of several milliseconds.

[0219] Next, in step S27, the signal processing unit 10 calculates heart rate variability (HRV). In step S27, the signal processing unit 10 may calculate power spectral density (PSD) by frequency analyzing the time series waveform of RRI. The frequency analysis of the time series waveform of RRI may use, for example, Welch's method.

[0220] More specifically, in step S27, the signal processing unit 10 calculates the time Δt estimated in step S26. ´´RRI The FFT and power spectrum may be calculated according to Welch's method, i.e., using window functions that overlap in time, thereby allowing the signal processing unit 10 to obtain the power spectral density (PSD) of the RRI.

[0221] 29 is a diagram showing an example of the power spectral density (PSD) of the RRI obtained by the signal processing unit 10. In FIG. 29, the horizontal axis represents frequency, and the vertical axis represents power spectral density (PSD). A well-known general method for calculating the power spectrum, such as Welch's method, may be used. Therefore, a detailed description of the method for calculating the power spectrum, such as Welch's method, will be omitted.

[0222] Fig. 29 is a diagram showing an example of the power spectral density calculated using Welch's method from the time-series waveform of the RRI shown in Fig. 28. According to the electronic device 1 of an embodiment, it is possible to calculate the power spectral density of the RRI based on the RRI with an accuracy of several ms. Therefore, according to the electronic device 1 of an embodiment, it is possible to accurately calculate the component of the heartbeat PSD between 0.15 Hz and 0.4 Hz, which is called the HF.

[0223] As described above, the electronic device 1 according to one embodiment can obtain, for example, detailed cardiac sound waveforms as shown in FIGS. 26 and 27, accurate RRI time series data as shown in FIG. 28, and the power spectral density (PSD) of the cardiac interval as shown in FIG. 29. The electronic device 1 according to one embodiment can detect the heartbeat of a human body or the like with high accuracy by transmitting and receiving radio waves. Therefore, the electronic device 1 according to one embodiment can detect weak vibrations such as the heartbeat of a human body with high accuracy by transmitting and receiving radio waves such as millimeter waves, and is expected to be useful in a wide variety of fields.

[0224] 26 to 29, only the first heart sound S1 and the second heart sound S2, which are prominent in healthy individuals, are described. However, according to the electronic device 1 of one embodiment, similar processing can be performed even when a third and / or fourth heart sound occurs due to an abnormal heartbeat.

[0225] (Other embodiments) Other embodiments will be described below.

[0226] In another embodiment, the electronic device 1 according to an embodiment may calculate the heart rate from the RRI data instead of obtaining the PSD from the RRI data. In this case, the signal processing unit 10 of the electronic device 1 may execute a process of calculating the heart rate from the RRI data, for example, in step S27 shown in Fig. 13. In this case, the signal processing unit 10 may execute the following processes 1 to 3 in step S27, for example. 1. Low-pass filter processing (passing frequencies below 1 Hz) 2. Reciprocal of RRI 3. Round the reciprocal taken in step 2 above to a natural number By these processes, the heart rate can be calculated from the RRI. In one embodiment, the electronic device 1 according to the embodiment may perform, for example, LPF processing, reciprocal calculation, and / or rounding to a natural number as the processing of step S27 shown in FIG.

[0227] In another embodiment, when performing the process shown in Fig. 13 and the above-described processes 1 to 3, the electronic device 1 according to an embodiment may change the process shown in step S24 shown in Fig. 13. For example, the electronic device 1 according to an embodiment may perform, as step S24 shown in Fig. 13, an autoencoder, template matching based on a cross-correlation function, classification and identification using an LSTM, and / or classification and identification of a multi-dimensional vector using an SVM.

[0228] In another embodiment, the electronic device 1 according to the embodiment may perform the process of step S25 in the process shown in Fig. 13 using the EM algorithm. However, the EM algorithm is not a Bayesian estimation method. Therefore, the EM algorithm does not have a mechanism for calculating a posterior distribution from a prior distribution, and when used to estimate a Gaussian mixture distribution, the variables π, μ, and Σ that describe the Gaussian mixture part are point-estimated. Therefore, when step S25 shown in FIG. 13 is executed using the EM algorithm, one of the following approaches is taken. For each time frame, Gaussian mixture estimation is performed using the EM algorithm, and the relationship between time frames is not taken into account. For each time frame, after estimating a Gaussian mixture distribution using the EM algorithm, you can take the moving average of the variables π, μ, and Σ of each Gaussian mixture distribution, or perform time series filtering using a Kalman filter.

[0229] In one embodiment, the transmitting antenna array 24 and / or the receiving antenna array 31 included in the electronic device 1 are not limited to the arrangement shown in Fig. 8. For example, in one embodiment, the receiving antenna array 31 included in the electronic device 1 may have a configuration as shown in Fig. 30. Fig. 30 is a diagram showing an example of a URA (Uniform Rectangular Array) receiving antenna. By employing a URA receiving antenna as shown in Fig. 30, it is possible to estimate the directions of arrival at two angles using only the URA receiving antenna, without changing the directivity using a beamformer in the transmitting antenna array 24.

[0230] 2, the signal processing unit 10 has been described as including functional units such as the heartbeat extraction unit 13 and the calculation unit 14. However, in one embodiment, the processing performed by the heartbeat extraction unit 13 and / or the calculation unit 14 may be performed by an external computer or processor.

[0231] In one embodiment, the process in step S14 shown in FIG. 12 (the process of analyzing singular value decomposition / principal components) may be substituted by another subspace method.

[0232] In one embodiment, the process in step S14 shown in Fig. 12 (the process of analyzing the singular value decomposition / principal components) may be omitted if the number of samples N per chirp can be set to a large number. In one embodiment, the process in step S14 shown in Fig. 12 (the process of analyzing the singular value decomposition / principal components) may also be omitted if the inclusion of other noise can be eliminated by a hardware technique or the like.

[0233] In the process of step S23 (generation of the heart sound envelope waveform) shown in Fig. 13, the results of the continuous wavelet transform are added in the frequency axis direction according to [Pseudo Code 2]. However, the envelope waveform may be generated by other methods, such as moving average or Hilbert transform.

[0234] An electronic device 1 according to an embodiment may calculate the RRI by taking the interval between negative peaks of the output of the reconstruction error of the autoencoder in the power spectral density (PSD) of the RRI shown in Fig. 29. That is, an electronic device 1 according to an embodiment may use a graph of Fig. 29 showing the reconstruction error of the autoencoder instead of the heart sound envelope shown in Fig. 24, and further set the point corresponding to the RRI as the negative peak in Fig. 29. In this case, the time resolution is limited by the window movement step in [Pseudo Code 3]. To reduce the time resolution, the window movement step in [Pseudo Code 3] may be reduced.

[0235] Figures 31 and 32 are diagrams illustrating negative peaks in the reconstruction error output of an autoencoder. Similar to Figure 21(A), Figure 31 is a diagram illustrating the MAPE of the best heart sound envelope. The horizontal axis of Figure 3 represents time in units of samples, and the vertical axis of Figure 31 represents MAPE (unit: %). Figure 32 is an enlarged view of the area enclosed by the dashed line in Figure 31. As shown in Figure 32, the graph showing the reconstruction error of the autoencoder reveals the negative peaks that occur when the heart sound template matches.

[0236] The electronic device 1 according to an embodiment may use the process in step S25 of FIG. 13 when estimating a Gaussian mixture distribution using the variational Bayes method. That is, the electronic device 1 according to an embodiment may use the process in step S25 of FIG. 13 only instead of the histograms of heart sound intervals shown in FIGS. 22 and 23. In this case, the Gaussian mixture distribution will be unimodal, i.e., a single Gaussian distribution. In this embodiment, the time intervals between the negative peaks in FIGS. 31 and 32 calculated using the above method may be converted into a histogram and used in the calculation in step S26 of FIG. 13. This time interval corresponds to only the heart sound interval. The histogram in this embodiment may have a single-hull characteristic. However, in this embodiment, if several noises are included, the variational Bayes method may be used to separate the Gaussian mixture distribution (the time between the negative peaks in FIG. 32 and the time intervals due to other noise peaks) to determine the desired time between the negative peaks in FIG. 32.

[0237] The electronic device 1 according to an embodiment may implement the process in step S25 of FIG. 13 using Markov Chain Monte Carlo methods (MCMC methods).

[0238] As described above, the electronic device 1 according to an embodiment detects weak vibrations such as heartbeats using, for example, a millimeter-wave sensor including multiple transmitting antennas and multiple receiving antennas. When the electronic device 1 according to an embodiment does not detect a target, it detects the target's body movement by changing the transmission phase of the antenna's beamforming pattern. On the other hand, when the electronic device 1 according to an embodiment detects the target's body movement, it performs beamforming in the direction of the body movement and detects the heartbeat. In this way, the electronic device 1 according to an embodiment can improve signal quality by automatically detecting the direction of the human body. Therefore, the electronic device 1 according to an embodiment can improve the heartbeat detection accuracy and / or detection range. Therefore, the electronic device 1 according to an embodiment can detect a human heartbeat with high accuracy.

[0239] While the present disclosure has been described based on various drawings and examples, it should be noted that those skilled in the art would easily be able to make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of the present disclosure. For example, the functions contained in each functional unit can be rearranged so as not to cause logical inconsistencies. Multiple functional units may be combined into one or divided. The above-described embodiments of the present disclosure are not limited to faithful implementation of each of the described embodiments, but may be implemented by combining features or omitting some features as appropriate. In other words, those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, these modifications and alterations are within the scope of the present disclosure. For example, in each embodiment, each functional unit, means, step, etc. can be added to other embodiments so as not to cause logical inconsistencies, or can be replaced with each functional unit, means, step, etc. of other embodiments. Furthermore, in each embodiment, multiple functional units, means, steps, etc. can be combined into one or divided into two or more. Furthermore, each of the above-described embodiments of the present disclosure is not limited to being implemented faithfully according to each of the described embodiments, but can also be implemented by combining each feature or omitting some of them as appropriate.

[0240] The above-described embodiment is not limited to being implemented only as the electronic device 1. For example, the above-described embodiment may be implemented as a control method for a device such as the electronic device 1. Furthermore, the above-described embodiment may be implemented as a program executed by a device such as the electronic device 1, or as a storage medium or recording medium on which a program is recorded.

[0241] The electronic device 1 according to the above-described embodiment has been described as including components constituting a so-called radar sensor, such as the transmitting antenna array 24 and the receiving antenna array 31. However, the electronic device according to an embodiment may be implemented as, for example, a configuration such as the signal processing unit 10. In this case, the signal processing unit 10 may be implemented as having a function of processing signals handled by, for example, the transmitting antenna array 24 and the receiving antenna array 31. [Explanation of symbols]

[0242] 1 Electronic equipment 10 Signal Processing Section 11 Signal generation processing section 12 Received signal processing section 13 Heart rate extraction unit 14 Calculation section 21 Transmit DAC 22 Transmitting circuit 23 Millimeter wave transmitter circuit 24 Transmitting Antenna Array 31 Receiving Antenna Array 32 Mixer 33 Receiving circuit 34 Receive ADC 50 Communication Interface 60 External equipment

Claims

1. a transmitting unit that transmits a transmission wave; a receiving unit that receives a reflected wave of the transmitted wave from a target; a signal processing unit that detects the distance, position, direction, and velocity of the target based on a converted signal obtained by Fourier transforming beat signals of the transmitted wave and the reflected wave; An electronic device comprising: the signal processing unit includes an extraction unit that extracts heartbeat information of the detected target based on the distance, direction, and speed of the target; The extraction unit extracting signal components corresponding to vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; applying a learning classifier to the extracted signal components to select a heart sound signal that is most appropriate as a heart sound; performing envelope processing on the most appropriate heart sound signal to select the most appropriate signal as a heartbeat; An electronic device that acquires a value indicating an interval between peak times from the optimum signal, and extracts the heartbeat interval of the target based on the variance and median value of the value indicating the interval.

2. The extraction unit extracting the signal components from a plurality of positions of the converted signal; The electronic device according to claim 1 , further comprising: a process for removing noise from the extracted signal components by using singular value decomposition, the process utilizing frequencies of the extracted signal components.

3. The extraction unit A process of applying a plurality of window functions to the converted signal to extract micro-Doppler components of the point cloud where the target exists; a process of performing at least one of principal component analysis and singular value decomposition on the extracted micro Doppler components; a process of performing frequency filtering using at least one of a short-time Fourier transform, a continuous wavelet transform, and a band-pass filter on a result of at least one of the principal component analysis and the singular value decomposition; The electronic device according to claim 1 , wherein the frequency-filtered result is subjected to multiresolution analysis including a discrete wavelet transform using a wavelet function and a scaling function appropriate for heart sounds.

4. the extraction unit uses a learning classifier to determine whether a signal component corresponding to vibrations associated with the heartbeat of the target is optimal as a heart sound signal; The electronic device according to claim 1 , wherein an envelope signal is extracted using a moving variance process for the signal component determined to be optimal by the learning classifier.

5. The extraction unit extracting an envelope signal from the signal components using any of continuous wavelet transform, discrete wavelet transform, wavelet scattering coefficients, mel-frequency cepstrum coefficients, or moving variance processing; The electronic device of claim 1 , wherein a trained learning classifier is used for the envelope signal to determine an optimal envelope signal based on scores calculated for the heart sounds of the signal components.

6. The extraction unit Obtaining interval values ​​at peak times for the extracted optimal envelope signal, and estimating the variance and center value of a set of interval values; generating a criterion for extracting cardiac intervals from the envelope signal based on the estimated variance and center value; The electronic device of claim 5 , wherein the criteria are used to extract cardiac intervals from the envelope signal.

7. The extraction unit Detecting a first heart sound and a second heart sound based on the extracted optimal envelope signal; 7. The electronic device according to claim 6, wherein the heart beat interval of the target is calculated by using DP matching based on a histogram or a probability density distribution of the interval between the first heart sound and the second heart sound and the interval between the second heart sound and the first heart sound.

8. transmitting a transmission wave from a transmitting unit; receiving a reflected wave of the transmitted wave from a target; detecting a distance position, a direction, and a velocity of the target based on a converted signal obtained by performing a Fourier transform on beat signals of the transmitted wave and the reflected wave; extracting heartbeat information of the detected target based on the distance, direction, and speed of the target; A control method for an electronic device, comprising: In the step of extracting heartbeat information of the target, extracting signal components corresponding to vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; applying a learning classifier to the extracted signal components to select a heart sound signal that is most appropriate as a heart sound; performing envelope processing on the most appropriate heart sound signal to select the most appropriate signal as a heartbeat; A control method for an electronic device, comprising: obtaining a value indicating an interval between peak times from the optimum signal; and extracting the heartbeat interval of the target based on the variance and median value of the value indicating the interval.

9. For electronic devices, transmitting a transmission wave from a transmitting unit; receiving a reflected wave of the transmitted wave from a target; detecting a distance position, a direction, and a velocity of the target based on a converted signal obtained by performing a Fourier transform on beat signals of the transmitted wave and the reflected wave; extracting heartbeat information of the detected target based on the distance, direction, and speed of the target; A program for executing In the step of extracting heartbeat information of the target, extracting signal components corresponding to vibrations caused by the heartbeat of the target from the converted signal using a plurality of window functions centered on a plurality of distance ranges; applying a learning classifier to the extracted signal components to select a heart sound signal that is most appropriate as a heart sound; performing envelope processing on the most appropriate heart sound signal to select the most appropriate signal as a heartbeat; A program that acquires a value indicating an interval between peak times from the optimum signal, and extracts the heartbeat interval of the target based on the variance and median value of the value indicating the interval.

Images