Electronic device, control method for electronic device, and program

Abstract

This electronic device comprises: a transmitting unit for transmitting a transmitted wave; a receiving unit for receiving a reflected wave from a target of the transmitted wave; and a signal processing unit for detecting a distance position, direction, and speed of the target on the basis of a converted signal obtained after a beat signal of the transmitted wave and the reflected wave has been subjected to a Fourier transform. The signal processing unit is provided with an extracting unit for extracting heartbeat information of the target on the basis of the detected distance, direction and speed of the target. The extracting unit: uses a plurality of window functions to extract, from the converted signal, signal components corresponding to vibrations that accompany a heartbeat of the target; applies a learning classifier to the extracted signal components to select a most appropriate heart sound signal as a heart sound; performs envelope processing with respect to the most appropriate heart sound signal to select a most appropriate optimal signal as the heartbeat; acquires values indicating intervals between peak times from the optimal signal; and extracts a heartbeat interval of the target on the basis of a variance and a central value of the values indicating the intervals.

Description

Cross - reference to related applications

[0001] This application claims the priority of Japanese Patent Application No. 2022 - 148632, filed in Japan on September 16, 2022, and incorporates the entire disclosure of the prior application herein by reference.

Technical Field

[0002] The present disclosure relates to an electronic device, a method for controlling an electronic device, and a program.

Background Art

[0003] For example, in fields such as industries related to automobiles, technologies for measuring the distance between a host vehicle and a predetermined object, etc. are highly regarded. In particular, in recent years, technologies for radar (Radio Detecting and Ranging) that measure the distance to an object by transmitting radio waves such as millimeter waves and receiving the reflected waves reflected from an object such as an obstacle have been variously studied. The importance of such technologies for measuring distances, etc. is expected to increase further in the future with the development of technologies for assisting a driver's driving and technologies related to autonomous driving that automate part or all of the driving.

[0004] Also, various proposals have been made for technologies for detecting the presence of an object, etc. by receiving the reflected waves reflected from a predetermined object by the transmitted radio waves. For example, Patent Document 1 proposes a device that can detect the presence of a person and the biometric information of a person by using microwaves. Also, for example, Patent Document 2 proposes a device that detects vital signs such as the frequency of a living body's respiration or heartbeat based on the reflected signal of a microwave radar.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

[0006] An electronic device according to an embodiment includes a transmission unit that transmits a transmission wave, a reception unit that receives a reflected wave from a target of the transmission wave, and a signal processing unit that detects a distance position, a direction, and a speed of the target based on a conversion signal obtained by performing a Fourier transform on a beat signal of the transmission wave and the reflected wave. The signal processing unit includes an extraction unit that extracts heartbeat information of the target based on the detected distance, direction, and speed of the target. The extraction unit extracts a signal component corresponding to vibrations associated with the heartbeat of the target from the conversion signal by using a plurality of window functions, selects the most appropriate heart sound signal as a heart sound by applying a learning classifier to the extracted signal component, , centered around multiple distance ranges, selects the most appropriate optimal signal as a heartbeat by performing an envelope process on the most appropriate heart sound signal, obtains a value indicating an interval of peak times from the optimal signal, and extracts a heartbeat interval of the target based on a variance and a central value of the value indicating the interval.

[0007] A control method of an electronic device according to an embodiment includes a step of transmitting a transmission wave from a transmission unit, a step of receiving, in reception, a reflected wave from a target of the transmission wave, a step of detecting a distance position, a direction, and a speed of the target based on a conversion signal obtained by performing a Fourier transform on a beat signal of the transmission wave and the reflected wave, and a step of extracting heartbeat information of the target based on the detected distance, direction, and speed of the target. In the step of extracting the heartbeat information of the target, a signal component corresponding to vibrations associated with the heartbeat of the target is extracted from the conversion signal by using a plurality of window functions, , centered around multiple distance ranges, ​ By applying a learning classifier to the extracted signal components, select the heart sound signal that is most appropriate as a heart sound, By performing envelope processing on the most appropriate heart sound signal, select the optimal signal that is most appropriate as a heartbeat, Obtain a value indicating the interval between peak times from the optimal signal, and extract the inter-beat interval of the target based on the variance and median of the values indicating the interval.

[0008] A program according to an embodiment causes an electronic device to transmit a transmission wave from a transmission unit, receive a reflected wave from the target of the transmission wave during reception, detect the distance position, direction, and speed of the target based on a conversion signal after performing Fourier transform on the beat signal of the transmission wave and the reflected wave, extract heartbeat information of the target based on the detected distance, direction, and speed of the target, and execute. In the step of extracting the heartbeat information of the target, from the conversion signal , centered around multiple distance ranges, extract signal components corresponding to vibrations associated with the heartbeat of the target using a plurality of window functions, By applying a learning classifier to the extracted signal components, select the heart sound signal that is most appropriate as a heart sound, By performing envelope processing on the most appropriate heart sound signal, select the optimal signal that is most appropriate as a heartbeat, Obtain a value indicating the interval between peak times from the optimal signal, and extract the inter-beat interval of the target based on the variance and median of the values indicating the interval.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Mode for Carrying Out the Invention

[0010] If a weak vibration such as a heartbeat of a human body or the like can be detected with good accuracy by transmitting and receiving radio waves such as millimeter waves, it can be expected to be useful in a wide variety of fields. An object of the present disclosure is to provide an electronic device, a control method for the electronic device, and a program capable of detecting a heartbeat of a human body or the like with good accuracy by transmitting and receiving radio waves. According to one embodiment, an electronic device, a control method for the electronic device, and a program capable of detecting a heartbeat of a human body or the like with good accuracy by transmitting and receiving radio waves can be provided. Hereinafter, one embodiment will be described in detail with reference to the drawings.

[0011] In the present disclosure, the “electronic device” may be a device driven by electric power. Also, the “user” may be a person (typically a human) or an animal who uses the system and / or the electronic device according to one embodiment. The user may include a person who monitors an object such as a human by using the electronic device according to one embodiment. Also, the “object” may be a person (for example, a human or an animal) who is the object to be monitored by the electronic device according to one embodiment. Further, the user may include the object.

[0012] In the present disclosure, the “heart beat” may be the pulsation of the heart. Also, the “pulsation” may be the rhythmic contraction movement performed by the heart.

[0013] Also, in the present disclosure, the “heart rate” refers to the number of times the heart pulsates within a certain period of time. For example, the heart rate may be the number of pulsations per minute. When the heart pumps blood, a pulsation occurs in the artery. Therefore, the number of pulsations of the artery may be referred to as the pulse rate or simply the pulse.

[0014] Furthermore, in the present disclosure, the "heart sound" may be the sound of the heartbeat of the heart. That is, the heart sound may be the sound that occurs when the heart contracts and expands. Here, the "heart sound" may consist of a low and long first sound based on the tension of the ventricular muscle, the closure of the mitral valve, the start of blood ejection into the artery, and / or the acceleration of blood flow, and a high and short second sound following this, which is derived from the closure of the aortic valve and / or the closure of the pulmonary valve.

[0015] In the present disclosure, the heart sound is not necessarily limited to a physical sound based on air vibration, and may also mean the vibration itself caused by the heartbeat (pulsation) of the heart. For example, in the present disclosure, the heart beat may imply the vibration source in some cases, and the heart sound may imply the vibration itself caused by the vibration source. Furthermore, in the present disclosure, the heart beat may imply the heart sound depending on the situation.

[0016] Also, in the present disclosure, the heart sound may be the vibration of the body accompanying the movement of the heart. The heart sound may be, for example, the vibration of the whole body, or a part of the body such as the head, neck, chest, throat, arms, legs, wrists, or other parts. The heart sound may be, for example, the vibration of the skin on the surface of the whole body, or a part of the body such as the head, neck, chest, throat, arms, legs, wrists, or other parts. The heart sound may be, for example, the vibration of clothing, underwear, glasses, or other worn items worn by the subject accompanying the vibration of the skin on the surface of the whole body, or a part of the body such as the head, neck, chest, throat, arms, legs, wrists, or other parts. The heart beat may be the pulsation of the heart itself. It may be assumed that the heart rate interval or heart rate, etc. is calculated from the movement of the heart beat. In the present disclosure, the pulsation of the heart may also be referred to as the heartbeat.

[0017] An electronic device according to an embodiment can detect the heartbeat of a target such as a person existing around the electronic device. Therefore, the scenes where the electronic device according to an embodiment is assumed to be used may be, for example, specific facilities used by those engaged in social activities, such as companies, hospitals, nursing homes, schools, sports gyms, and nursing facilities. For example, in a company, it is extremely important to grasp and / or manage the health status of employees and the like. Similarly, in a hospital, it is extremely important to grasp and / or manage the health status of patients and medical staff, etc., and in a nursing home, it is extremely important to grasp and / or manage the health status of residents and staff, etc. The scenes where the electronic device according to an embodiment is used are not limited to the above-mentioned facilities such as companies, hospitals, and nursing homes, but may be any facility where grasping and / or managing the health status of the target is desired. The arbitrary facility may include, for example, a non-commercial facility such as the user's home. Also, the scenes where the electronic device according to an embodiment is used are not limited to indoors, but may also be outdoors. For example, the scenes where the electronic device according to an embodiment is used may be inside a moving body such as a train, a bus, and an airplane, as well as stations and boarding areas. Also, as the scenes where the electronic device according to an embodiment is used, a moving body such as an automobile, an airplane, or a ship, a hotel, the user's home, a living room at home, a bathroom, a toilet, or a bedroom may be used.

[0018] An electronic device according to an embodiment may be used, for example, in a nursing facility or the like for detecting or monitoring the heartbeat of a target such as a person requiring nursing care or a person requiring care. Further, when an abnormality is recognized in the heartbeat of a target such as a person requiring nursing care or a person requiring care, the electronic device according to an embodiment may issue a predetermined warning to, for example, the person himself / herself and / or another person. Therefore, according to the electronic device according to an embodiment, for example, the person himself / herself and / or the staff of a nursing facility or the like can recognize that an abnormality is recognized in the pulse of a target such as a person requiring nursing care or a person requiring care. On the other hand, when no abnormality is recognized (for example, it is recognized as normal) in the heartbeat of a target such as a person requiring nursing care or a person requiring care, the electronic device according to an embodiment may notify the person himself / herself and / or another person to that effect. Therefore, according to the electronic device according to an embodiment, for example, the person himself / herself and / or the staff of a nursing facility or the like can recognize that the pulse of a target such as a person requiring nursing care or a person requiring care is normal.

[0019] Further, the electronic device according to an embodiment may detect the pulse of other animals other than humans. The electronic device according to an embodiment described below will be described as an example of detecting the pulse of a human by a sensor based on a technology such as a millimeter-wave radar.

[0020] The electronic device according to an embodiment may be installed on any stationary object or any moving object. The electronic device according to an embodiment can transmit a transmission wave from a transmission antenna to the surroundings of the electronic device. Further, the electronic device according to an embodiment can receive a reflected wave obtained by reflecting the transmission wave from a reception antenna. At least one of the transmission antenna and the reception antenna may be provided in the electronic device or may be provided, for example, in a radar sensor or the like.

[0021] Hereinafter, as a typical example, the electronic device according to one embodiment will be described as being stationary. On the other hand, the object (human) to which the electronic device according to one embodiment detects the pulse may be stationary, may be moving, or may be moving the body in a stationary state. The electronic device according to one embodiment can measure the distance between the electronic device and the object, etc. in a situation where the objects around the electronic device can move, similar to a normal radar sensor. Also, the electronic device according to one embodiment can measure the distance between the electronic device and the object, etc. even when both the electronic device and the object are stationary.

[0022] The electronic device according to one embodiment will be described in detail below with reference to the drawings. First, an example of object detection by the electronic device according to one embodiment will be described.

[0023] FIG. 1 is a diagram for explaining an example of the usage mode of the electronic device according to one embodiment. FIG. 1 shows an example of an electronic device having the functions of a sensor including a transmission antenna and a reception antenna according to one embodiment.

[0024] As shown in FIG. 1, the electronic device 1 according to one embodiment may include a transmission unit and a reception unit to be described later. As will be described later, the transmission unit may include a transmission antenna array 24. Also, the reception unit may include a reception antenna array 31. The specific configurations of the electronic device 1, the transmission unit, and the reception unit will be described later. FIG. 1 shows the situation where the electronic device 1 includes the transmission antenna array 24 and the reception antenna array 31 for ease of viewing. Also, the electronic device 1 may appropriately include at least a part of other functional units such as at least a part of the signal processing unit 10 (FIG. 2) included in the electronic device 1. Also, the electronic device 1 may provide at least a part of other functional units such as at least a part of the signal processing unit 10 (FIG. 2) included in the electronic device 1 outside the electronic device 1. In FIG. 1, the electronic device 1 may be moving, but may be stationary without moving.

[0025] In the example shown in FIG. 1, the electronic device 1 is shown in a simplified manner, including a transmitting unit having a transmitting antenna array 24 and a receiving unit having a receiving antenna array 31. The electronic device 1 may include, for example, a plurality of transmitting units and / or a plurality of receiving units. The transmitting unit may include a transmitting antenna array 24 composed of a plurality of transmitting antennas. Further, the receiving unit may include a receiving antenna array 31 composed of a plurality of receiving antennas. Here, the positions where the transmitting unit and / or the receiving unit are installed in the electronic device 1 are not limited to the positions shown in FIG. 1, and may be other positions as appropriate. Also, the number of the transmitting unit and / or the receiving unit may be any number of one or more according to various conditions (or requirements) such as the detection range and / or detection accuracy of the heartbeat by the electronic device 1.

[0026] As will be described later, the electronic device 1 transmits electromagnetic waves as transmission waves from the transmitting antenna array 24. For example, when a predetermined object (e.g., the target 200 shown in FIG. 1) exists around the electronic device 1, at least a part of the transmission waves transmitted from the electronic device 1 is reflected by the object to become reflected waves. Then, by receiving such reflected waves with, for example, the receiving antenna array 31 of the electronic device 1, the electronic device 1 can detect the object as a target.

[0027] The electronic device 1 including the transmitting antenna array 24 may typically be a radar (Radio Detecting and Ranging) sensor that transmits and receives radio waves. However, the electronic device 1 is not limited to a radar sensor. The electronic device 1 according to one embodiment may be, for example, a sensor based on the technology of LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) using light waves. Such sensors can be configured to include, for example, patch antennas. Since technologies such as RADAR and LIDAR are already known, detailed descriptions may be appropriately simplified or omitted. Further, the electronic device 1 according to one embodiment may be, for example, a sensor based on a technology that transmits and receives sound waves or ultrasonic waves to detect an object.

[0028] The electronic device 1 shown in FIG. 1 receives the reflected wave of the transmitted wave transmitted from the transmission antenna array 24 from the reception antenna array 31. In this way, the electronic device 1 can detect a predetermined target 200 existing within a predetermined distance from the electronic device 1. For example, as shown in FIG. 1, the electronic device 1 can measure the distance L between the electronic device 1 and the predetermined target 200. Further, the electronic device 1 can also measure the relative speed between the electronic device 1 and the predetermined target 200. Furthermore, the electronic device 1 can also measure the direction (arrival angle θ) in which the reflected wave from the predetermined target 200 arrives at the electronic device 1.

[0029] In FIG. 1, the XY plane may be, for example, a plane substantially parallel to the ground surface. In this case, the positive direction of the Z axis shown in FIG. 1 may indicate vertically upward. In FIG. 1, the electronic device 1 may be arranged on a plane parallel to the XY plane. Also, in FIG. 1, the target 200 may be, for example, in a state of standing on the ground surface substantially parallel to the XY plane.

[0030] Here, the target 200 may be, for example, a human being existing around the electronic device 1 or the like. Also, the target 200 may be, for example, a living being other than a human being, such as an animal existing around the electronic device 1. As described above, the target 200 may be moving or may be stopped or stationary. In the present disclosure, the object detected by the electronic device 1 includes not only inanimate objects such as arbitrary objects, but also living beings such as humans, dogs, cats, horses, and other animals. The object detected by the electronic device 1 of the present disclosure may include targets such as humans, objects, and animals detected by radar technology. In the present disclosure, the target may include humans, objects, animals, and the like. Hereinafter, an object such as the target 200 existing around the electronic device 1 will be described assuming that it is a human (or animal). Hereinafter, the “target 200” will also be referred to as the “subject 200” as appropriate. In the present disclosure, the target may be the above-described target 200.

[0031] In FIG. 1, the ratio between the size of the electronic device 1 and the size of the object 200 does not necessarily indicate the actual ratio. Also, in FIG. 1, the transmission antenna array 24 of the transmission unit and the reception antenna array 31 of the reception unit are shown in a state of being installed outside the electronic device 1. However, in one embodiment, the transmission antenna array 24 of the transmission unit and / or the reception antenna array 31 of the reception unit may be installed at various positions of the electronic device 1. For example, in one embodiment, the transmission antenna array 24 of the transmission unit and / or the reception antenna array 31 of the reception unit may be installed inside the electronic device 1 so as not to appear on the exterior of the electronic device 1.

[0032] Hereinafter, as a typical example, the transmission antenna of the electronic device 1 will be described as transmitting radio waves in a frequency band such as millimeter waves (30 GHz or higher) or quasi-millimeter waves (for example, around 20 GHz to 30 GHz). On the other hand, the transmission antenna of the electronic device 1 may transmit radio waves having a frequency bandwidth of 4 GHz, such as 77 GHz to 81 GHz.

[0033] FIG. 2 is a functional block diagram schematically showing a configuration example of the electronic device 1 according to one embodiment. Hereinafter, an example of the configuration of the electronic device 1 according to one embodiment will be described.

[0034] When measuring distance and the like by a millimeter-wave radar, a frequency-modulated continuous-wave radar (hereinafter, referred to as an FMCW radar (Frequency Modulated Continuous Wave radar)) is often used. In the FMCW radar, a transmission signal is generated by sweeping the frequency of the transmitted radio wave. Therefore, for example, in a millimeter-wave FMCW radar using radio waves in the 79 GHz frequency band, the frequency of the radio wave to be used has a frequency bandwidth of 4 GHz, such as 77 GHz to 81 GHz. The radar in the 79 GHz frequency band has a feature that the available frequency bandwidth is wider than that of other millimeter-wave / quasi-millimeter-wave radars such as the 24 GHz, 60 GHz, and 76 GHz frequency bands. Hereinafter, such an embodiment will be described as an example.

[0035] The FMCW radar method used in the present disclosure may include the FCM method (Fast-Chirp Modulation) that transmits a chirp signal at a cycle shorter than normal. The signal generated by the electronic device 1 is not limited to the signal of the FMCW method. The signal generated by the electronic device 1 may be signals of various methods other than the FMCW method. The transmission signal sequence stored in any storage unit may be different depending on these various methods. For example, in the case of the above-described FMCW radar signal, signals with increasing and decreasing frequencies may be used for each time sample. Since the above-described various methods can appropriately apply known techniques, a more detailed description is omitted.

[0036] As shown in FIG. 2, the electronic device 1 according to an embodiment includes a signal processing unit 10. The signal processing unit 10 may include a signal generation processing unit 11, a received signal processing unit 12, a heartbeat extraction unit 13, and a calculation unit 14. The heartbeat extraction unit 13 may execute, for example, a process of extracting a micro-Doppler component. Further, the heartbeat extraction unit 13 may execute a process of extracting an envelope of the heart sound of the subject 200. The calculation unit 14 may execute, for example, a process of calculating the heart rate interval (RRI) of the subject 200. Further, the calculation unit 14 may execute, for example, a process of calculating the heartbeat of the subject 200. Furthermore, the calculation unit 14 may execute, for example, a process of calculating the heart rate variability (HRV) of the subject 200. In this case, the calculation unit 14 may execute a process of performing frequency analysis on the time-series data of the extracted heart rate interval of the subject 200. Further, the calculation unit 14 may execute a process of calculating the heart rate variability of the subject 200 based on the frequency analysis of the time-series data of the heart rate interval. The signal generation processing unit 11, the received signal processing unit 12, the heartbeat extraction unit 13, and the calculation unit 14 will be described later as appropriate. In the present disclosure, the heart sound may be, for example, a chest vibration waveform directly observed by radar (see FIG. 20, etc.), or may be a chest vibration. The heartbeat is the beating of the heart itself. It may be assumed that the heart rate interval, heart rate, etc. are calculated from the movement of the heartbeat. The heart rate interval may be a time interval between the beating of the heart and the next beating of the heart.

[0037] Also, the electronic device 1 according to one embodiment includes, as a transmission unit, a transmission DAC 21, a transmission circuit 22, a millimeter-wave transmission circuit 23, and a transmission antenna array 24. Further, the electronic device 1 according to one embodiment includes, as a reception unit, a reception antenna array 31, a mixer 32, a reception circuit 33, and a reception ADC 34. The electronic device 1 according to one embodiment may not include at least any of the functional units shown in FIG. 2, or may include functional units other than the functional units shown in FIG. 2. The electronic device 1 shown in FIG. 2 may be configured using a circuit configured in basically the same manner as a general radar using electromagnetic waves in a millimeter-wave band or the like. On the other hand, in the electronic device 1 according to one embodiment, the signal processing by the signal processing unit 10 may include processing different from that of a conventional general radar.

[0038] The signal processing unit 10 included in the electronic device 1 according to one embodiment can control the operation of the entire electronic device 1, including the control of each functional unit constituting the electronic device 1. In particular, the signal processing unit 10 performs various processes on the signals handled by the electronic device 1. The signal processing unit 10 may include at least one processor, such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), to provide control and processing capabilities for executing various functions. The signal processing unit 10 may be realized by a single processor, or may be realized by several processors, or may be realized by individual processors. The processor may be realized as a single integrated circuit. An integrated circuit is also referred to as an IC (Integrated Circuit). The processor may be realized as a plurality of communicably connected integrated circuits and discrete circuits. The processor may be realized based on various other known technologies. In one embodiment, the signal processing unit 10 may be configured, for example, as a CPU (hardware) and a program (software) executed by the CPU. The signal processing unit 10 may appropriately include a storage unit (memory) necessary for the operation of the signal processing unit 10.

[0039] The signal generation processing unit 11 of the signal processing unit 10 generates a signal to be transmitted from the electronic device 1. In the electronic device 1 according to an embodiment, the signal generation processing unit 11 may generate a transmission signal (transmission chirp signal) such as a chirp signal, for example. In particular, the signal generation processing unit 11 may generate a signal whose frequency linearly changes periodically (linear chirp signal). For example, the signal generation processing unit 11 may generate a chirp signal whose frequency periodically and linearly increases from 77 GHz to 81 GHz as time elapses. Also, for example, the signal generation processing unit 11 may generate a signal whose frequency periodically repeats linear increase (up-chirp) and decrease (down-chirp) from 77 GHz to 81 GHz as time elapses. The signal generated by the signal generation processing unit 11 may be preset, for example, in the signal processing unit 10. Also, the signal generated by the signal generation processing unit 11 may be pre-stored, for example, in an arbitrary storage unit in the signal processing unit 10 or the like. Since chirp signals used in technical fields such as radar are known, more detailed descriptions will be appropriately simplified or omitted. The signal generated by the signal generation processing unit 11 is supplied to the transmission DAC 21. Therefore, the signal generation processing unit 11 may be connected to the transmission DAC 21.

[0040] The transmission DAC (Digital-Analog Converter) 21 has a function of converting the digital signal supplied from the signal generation processing unit 11 into an analog signal. The transmission DAC 21 may be configured to include a general digital-analog converter. The signal analogized by the transmission DAC 21 is supplied to the transmission circuit 22. Therefore, the transmission DAC 21 may be connected to the transmission circuit 22.

[0041] The transmission circuit 22 has a function of converting the signal analogized by the transmission DAC 21 into a band of an intermediate frequency (IF). The transmission circuit 22 may be configured to include a general IF band transmission circuit. The signal processed by the transmission circuit 22 is supplied to the millimeter wave transmission circuit 23. Therefore, the transmission circuit 22 may be connected to the millimeter wave transmission circuit 23.

[0042] The millimeter-wave transmission circuit 23 has a function of transmitting the signal processed by the transmission circuit 22 as a millimeter wave (RF wave). The millimeter-wave transmission circuit 23 may be configured to include a general millimeter-wave transmission circuit. The signal processed by the millimeter-wave transmission circuit 23 is supplied to the transmission antenna array 24. For this reason, the millimeter-wave transmission circuit 23 may be connected to the transmission antenna array 24. Also, the signal processed by the millimeter-wave transmission circuit 23 is also supplied to the mixer 32. For this reason, the millimeter-wave transmission circuit 23 may also be connected to the mixer 32.

[0043] The transmission antenna array 24 is formed by arranging a plurality of transmission antennas in an array. In FIG. 2, the configuration of the transmission antenna array 24 is shown in a simplified manner. The transmission antenna array 24 transmits the signal processed by the millimeter-wave transmission circuit 23 to the outside of the electronic device 1. The transmission antenna array 24 may be configured to include a transmission antenna array used in a general millimeter-wave radar.

[0044] In this way, the electronic device 1 according to one embodiment includes a transmission antenna (transmission antenna array 24), and can transmit a transmission signal (for example, a transmission chirp signal) as a transmission wave from the transmission antenna array 24.

[0045] For example, as shown in FIG. 2, assume a case where an object such as a subject 200 exists around the electronic device 1. In this case, at least a part of the transmission wave transmitted from the transmission antenna array 24 is reflected by an object such as the subject 200. At least a part of the transmission wave transmitted from the transmission antenna array 24 that is reflected by an object such as the subject 200 can be reflected toward the reception antenna array 31.

[0046] The reception antenna array 31 receives the reflected wave. Here, the reflected wave may be at least a part of the transmission wave transmitted from the transmission antenna array 24 that is reflected by an object such as the subject 200.

[0047] The receiving antenna array 31 is formed by arranging a plurality of receiving antennas in an array. In FIG. 2, the configuration of the receiving antenna array 31 is shown in a simplified manner. The receiving antenna array 31 receives the reflected wave of the transmitted wave transmitted from the transmitting antenna array 24. The receiving antenna array 31 may be configured to include a receiving antenna array used in a general millimeter-wave radar. The receiving antenna array 31 supplies the received signal received as the reflected wave to the mixer 32. For this reason, the receiving antenna array 31 may be connected to the mixer 32.

[0048] The mixer 32 converts the signal (transmission signal) processed by the millimeter-wave transmission circuit 23 and the received signal received by the receiving antenna array 31 into an intermediate frequency (IF) band. The mixer 32 may be configured to include a mixer used in a general millimeter-wave radar. The mixer 32 supplies the signal generated as a synthesized result to the receiving circuit 33. For this reason, the mixer 32 may be connected to the receiving circuit 33.

[0049] The receiving circuit 33 has a function of analog-processing the signal converted into the IF band by the mixer 32. The receiving circuit 33 may be configured to include a receiving circuit that converts to a general IF band. The signal processed by the receiving circuit 33 is supplied to the receiving ADC 34. For this reason, the receiving circuit 33 may be connected to the receiving ADC 34.

[0050] The receiving ADC (analog-digital converter) 34 has a function of converting the analog signal supplied from the receiving circuit 33 into a digital signal. The receiving ADC 34 may be configured to include a general analog-digital converter. The signal digitized by the receiving ADC 34 is supplied to the receiving signal processing unit 12 of the signal processing unit 10. For this reason, the receiving ADC 34 may be connected to the signal processing unit 10.

[0051] The reception signal processing unit 12 of the signal processing unit 10 has a function of performing various processes on the digital signal supplied from the reception DAC 34. For example, the reception signal processing unit 12 calculates the distance from the electronic device 1 to an object such as the subject 200 (distance measurement) based on the digital signal supplied from the reception DAC 34. Also, the reception signal processing unit 12 calculates the relative speed of an object such as the subject 200 with respect to the electronic device 1 based on the digital signal supplied from the reception DAC 34 (speed measurement). Further, the reception signal processing unit 12 calculates the azimuth angle of an object such as the subject 200 as seen from the electronic device 1 based on the digital signal supplied from the reception DAC 34 (angle measurement). Specifically, I / Q converted data may be input to the reception signal processing unit 12. When such data is input, the reception signal processing unit 12 performs two-dimensional fast Fourier transforms (2D-FFT) in the distance (Range) direction and the speed (Velocity) direction, respectively. Then, the reception signal processing unit 12 suppresses false alarms and makes the probability constant by removing noise points by a process such as CFAR (Constant False Alarm Rate). And the reception signal processing unit 12 obtains the position of an object such as the subject 200 by performing arrival angle estimation on points that satisfy the CFAR criterion. The information generated as a result of distance measurement, speed measurement, and angle measurement by the reception signal processing unit 12 may be supplied to the heartbeat extraction unit 13.

[0052] The heartbeat extraction unit 13 extracts information regarding the heartbeat from the information generated by the reception signal processing unit 12. The operation of extracting information regarding the heartbeat by the heartbeat extraction unit 13 will be described in further detail later. The information regarding the heartbeat extracted by the heartbeat extraction unit 13 may be supplied to the calculation unit 14.

[0053] The calculation unit 14 performs various calculation processes and / or arithmetic operations on the information regarding the heartbeat supplied from the heartbeat extraction unit 13. The various calculation processes and / or arithmetic operations by the calculation unit 14 will be described in further detail later. The various information calculated and / or processed by the calculation unit 14 may be supplied to, for example, the communication interface 50. For this reason, the calculation unit 14 and / or the signal processing unit 10 may be connected to the communication interface 50. The various information calculated and / or processed by the calculation unit 14 may be supplied to other functional units other than the communication interface 50.

[0054] The communication interface 50 is configured to include an interface that outputs the information supplied from the signal processing unit 10 to, for example, an external device 60. The communication interface 50 may output information on at least any one of the position, speed, and angle of an object such as the subject 200 to an external device 60 as a signal such as, for example, CAN (Controller Area Network). For example, information on at least any one of the position, speed, and angle of an object such as the subject 200 may be supplied to an external device 60 or the like via the communication interface 50. For this reason, the communication interface 50 may be connected to an external device 60 or the like.

[0055] As shown in FIG. 2, the electronic device 1 according to one embodiment may be connected to an external device 60 by wire or wirelessly via the communication interface 50. In one embodiment, the external device 60 may be configured to include any computer and / or any control device or the like. Also, the electronic device 1 according to one embodiment may be configured to include the external device 60. The external device 60 can have various configurations according to the mode in which the information on the heartbeat and / or heart sound detected by the electronic device 1 is used. Therefore, a more detailed description of the external device 60 will be omitted.

[0056] FIG. 3 is a diagram for explaining an example of a chirp signal generated by the signal generation processing unit 11 of the signal processing unit 10.

[0057] FIG. 3 shows the time structure of one frame when the FCM (Fast-Chirp Modulation) method is used. FIG. 3 shows an example of a received signal of the FCM method. FCM is a method of repeating chirp signals shown as c1, c2, c3, c4, …, cn in FIG. 3 at short intervals (for example, longer than the round-trip time between the radar and the target of the electromagnetic wave calculated from the maximum ranging distance). In FCM, for the convenience of signal processing of the received signal, it is often divided into sub-frame units as shown in FIG. 3 and the transmission and reception processing is performed.

[0058] In FIG. 3, the horizontal axis represents the elapsed time and the vertical axis represents the frequency. In the example shown in FIG. 3, the signal generation processing unit 11 generates a linear chirp signal whose frequency changes linearly periodically. In FIG. 3, each chirp signal is shown as c1, c2, c3, c4, …, cn. As shown in FIG. 3, in each chirp signal, the frequency increases linearly with the passage of time.

[0059] In the example shown in FIG. 3, several chirp signals such as c1, c2, c3, c4, …, cn are included as one sub-frame. That is, the sub-frame 1 and sub-frame 2 shown in FIG. 3 are each composed of several chirp signals such as c1, c2, c3, c4, …, cn. Also, in the example shown in FIG. 3, several sub-frames such as sub-frame 1, sub-frame 2, …, sub-frame N are included as one frame (one frame). That is, the one frame shown in FIG. 3 is composed of N sub-frames. Also, taking the one frame shown in FIG. 3 as frame 1, then frame 2, frame 3, … etc. may follow. These frames may each be composed of N sub-frames in the same way as frame 1. Also, a frame interval of a predetermined length may be included between the frames. One frame shown in FIG. 3 may have a length of about 30 milliseconds to 50 milliseconds, for example.

[0060] In the electronic device 1 according to one embodiment, the signal generation processing unit 11 may generate a transmission signal as any number of frames. Also, in FIG. 3, some of the chirp signals are shown omitted. Thus, the relationship between the time and frequency of the transmission signal generated by the signal generation processing unit 11 may be stored, for example, in the storage unit of the signal processing unit 10 or the like.

[0061] Thus, the electronic device 1 according to one embodiment may transmit a transmission signal composed of sub-frames including a plurality of chirp signals. Also, the electronic device 1 according to one embodiment may transmit a transmission signal composed of frames including a predetermined number of sub-frames.

[0062] Hereinafter, the electronic device 1 will be described as transmitting a transmission signal having a frame structure as shown in FIG. 3. However, the frame structure as shown in FIG. 3 is an example, and for example, the chirp signals included in one sub-frame may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate a sub-frame including an arbitrary number (for example, an arbitrary plurality) of chirp signals. Also, the sub-frame structure as shown in FIG. 3 is an example, and for example, the sub-frames included in one frame may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate a frame including an arbitrary number (for example, an arbitrary plurality) of sub-frames. The signal generation processing unit 11 may generate signals having different frequencies. The signal generation processing unit 11 may generate a plurality of discrete signals having different bandwidths with each frequency f being different.

[0063] FIG. 4 is a diagram showing a part of the sub-frame shown in FIG. 3 in another aspect. FIG. 4 shows each sample of the received signal obtained by receiving the transmission signal shown in FIG. 3 as a result of performing 2D-FFT (Two Dimensional Fast Fourier Transform), which is the processing performed in the reception signal processing unit 12 (FIG. 2) of the signal processing unit 10.

[0064] As shown in FIG. 4, in each sub-frame such as sub-frame 1, …, sub-frame N, each chirp signal c1, c2, c3, c4, …, cn is stored. In FIG. 4, each chirp signal c1, c2, c3, c4, …, cn is composed of each sample indicated by the cells arranged horizontally. The received signal shown in FIG. 4 is subjected to 2D-FFT, CFAR, and / or integrated signal processing of each sub-frame by the received signal processing unit 12 shown in FIG. 2.

[0065] FIG. 5 is a diagram showing an example in which a point group on a range-Doppler (distance-velocity) plane is calculated as a result of performing 2D-FFT, CFAR, and integrated signal processing of each sub-frame in the received signal processing unit 12 shown in FIG. 2.

[0066] In FIG. 5, the horizontal direction represents the range (distance), and the vertical direction represents the velocity. The filled cell s1 shown in FIG. 5 indicates a point group of signals that exceed the threshold processing of CFAR. The unfilled cell s2 shown in FIG. 5 indicates a bin (2D-FFT sample) without a point group that did not exceed the threshold of CFAR. The point group on the range-Doppler plane calculated in FIG. 5 has its azimuth from the radar calculated by direction estimation, and the position and velocity on the two-dimensional plane are calculated as a point group indicating an object such as the subject 200. Here, the direction estimation may be calculated by a beamformer and / or a subspace method. Representative subspace method algorithms include MUSIC (MUltiple SIgnal Classification), and ESPRIT (Estimation of Signal Parameters via Rotation Invariance Technique), etc.

[0067] FIG. 6 is a diagram showing an example of the result of the conversion of the point group coordinates from the range-Doppler plane shown in FIG. 5 to the XY plane after the reception signal processing unit 12 performs direction estimation. As shown in FIG. 6, the reception signal processing unit 12 can plot the point group PG on the XY plane. Here, the point group PG is composed of each point P. Further, each point P has an angle θ and a radial velocity Vr in polar coordinates.

[0068] The reception signal processing unit 12 detects an object existing in the range where the transmission wave T is transmitted based on at least any one of the results of 2D-FFT and angle estimation. The reception signal processing unit 12 may perform object detection, for example, by performing clustering processing based on the respectively estimated distance information, velocity information, and angle information. As an algorithm used when clustering data, for example, DBSCAN (Density-based spatial clustering of applications with noise) is known. This is an algorithm that performs density-based clustering. In the clustering process, for example, the average power of the points constituting the detected object may be calculated. The distance information, velocity information, angle information, and power information of the object detected by the reception signal processing unit 12 may be supplied to an external device 60 or the like, for example, via the communication interface 50.

[0069] As described above, the electronic device 1 may include a transmission antenna (transmission antenna array 24), a reception antenna (reception antenna array 31), and a signal processing unit 10. The transmission antenna array 24 transmits the transmission wave T. The reception antenna array 31 receives the reflected wave R obtained by reflecting the transmission wave T. Then, the signal processing unit 10 detects an object (such as an object like the subject 200) that reflects the transmission wave T based on the transmission signal transmitted as the transmission wave T and the reception signal received as the reflected wave R.

[0070] Next, the direction estimation of the incoming wave by the antenna array of the electronic device 1 according to an embodiment will be further described.

[0071] FIG. 7 is a diagram for explaining the configuration of the reception antenna array 31 of the electronic device 1 according to an embodiment and the principle of direction estimation of an incoming wave by the reception antenna array 31. FIG. 7 shows an example of reception of radio waves by the reception antenna array 31.

[0072] As shown in FIG. 7, the reception antenna array 31 may be formed by arranging sensors such as reception antennas in a straight line. As shown in FIG. 7, in one embodiment, the reception antenna array 31 may be configured to include a plurality of reception antennas arranged in a straight line. In FIG. 7, the reception antenna array 31 is shown by small circles for a plurality of antennas such as antenna x1, x2, x3,..., x M . The reception antenna array 31 may be configured by any plurality of antennas. Also, as shown in FIG. 7, the plurality of antennas constituting the reception antenna array 31 are assumed to be arranged at intervals of an array pitch d. In this way, a sensor array in which sensors (such as antennas, ultrasonic transducers, and microphones) corresponding to various physical waves are arranged in an array is also called a Uniform Linear Array (ULA). As shown in FIG. 7, physical waves (such as electromagnetic waves and sound waves) arrive from various directions such as θ1 and θ2, for example. Here, θ1 and θ2 may be the incoming angles described above. In this way, a sensor array such as the reception antenna array 31 can estimate the incoming direction (incoming angle) by using the phase difference generated in the measurement values between the sensors according to the incoming direction of the physical wave. In this way, the method of estimating the incoming direction of the wave is also referred to as incoming angle estimation or Direction of Arrival (DoA).

[0073] In the electronic device 1 according to one embodiment, at least one of the transmission antenna array 24 and the reception antenna array 31 may be configured by arranging a plurality of antennas linearly. Thereby, for example, in a millimeter-wave radar, the directivity during transmission and reception of radio waves can be appropriately narrowed. When transmitting a transmission wave, the direction of the transmission beam is often controlled by a beamformer. On the other hand, when receiving a reflected wave, the arrival direction of the reflected wave is often estimated by a subspace method (such as MUSIC and ESPRIT described above) rather than a beamformer. In the beamformer and the subspace method, in a ULA as shown in FIG. 7, for electromagnetic waves arriving from various directions, a phase difference occurs in the measurement values between sensors according to the arrival direction. Therefore, the arrival direction of the reflected wave can be estimated using the phase difference.

[0074] Next, the two-direction angle estimation of the arrival wave by the antenna array of the electronic device 1 according to one embodiment will be further described.

[0075] FIG. 8 is a diagram showing an example of the arrangement of antennas for estimating the arrival directions of two orthogonal angles.

[0076] As shown in FIG. 8, in the electronic device 1 according to one embodiment, the transmission antenna array 24 and / or the reception antenna array 31 may be configured to include an array of a plurality of patch antenna units.

[0077] In the transmission antenna array 24 shown in FIG. 8, one patch antenna unit may be configured to include a plurality of elements electrically connected in the direction of direction 1 shown in the figure. In each patch antenna unit, the plurality of elements may be electrically connected by wiring such as a strip line on a substrate. In each patch antenna unit, each of the plurality of elements may be spaced apart by a distance d shorter than half of the wavelength λ of the transmission wave. 1,t In FIG. 8, each patch antenna unit may be configured by electrically connecting two or more arbitrary numbers of elements.

[0078] Also, as shown in FIG. 8, the transmission antenna array 24 may be arranged by arraying a plurality of patch antenna units in the direction 2 shown in the figure. Each patch antenna unit may be arranged at an interval d shorter than half of the wavelength λ of the transmission wave. 2,t In one embodiment, the transmission antenna array 24 may include any number of two or more patch antenna units.

[0079] As shown in FIG. 8, in one embodiment, the reception antenna array 31 may be obtained by changing the arrangement of a plurality of elements in the transmission antenna array 24. That is, in the reception antenna array 31 shown in FIG. 8, one patch antenna unit may be configured to include a plurality of elements electrically connected in the direction 2 shown in the figure. In each patch antenna unit, the plurality of elements may be electrically connected by wiring such as a strip line on a substrate, for example. In each patch antenna unit, each of the plurality of elements may be arranged at an interval d shorter than half of the wavelength λ of the transmission wave. 2,s In FIG. 8, each patch antenna unit may be configured by electrically connecting any number of two or more elements.

[0080] Also, as shown in FIG. 8, the reception antenna array 31 may be arranged by arraying a plurality of patch antenna units in the direction 1 shown in the figure. Each patch antenna unit may be arranged at an interval d shorter than half of the wavelength λ of the transmission wave. 1,s In one embodiment, the reception antenna array 31 may include any number of two or more patch antenna units.

[0081] All the elements included in the transmission antenna array 24 and the reception antenna array 31 may be arranged on the same plane (for example, on the surface layer of the same substrate). Also, the transmission antenna array 24 and the reception antenna array 31 may be arranged close to each other (monostatic). Further, the direction 1 and the direction 2 shown in FIG. 8 may be geometrically orthogonal.

[0082] The directivities of the transmitting antenna and the receiving antenna can be appropriately narrowed by the transmitting antenna array 24 and the receiving antenna array 31 as shown in FIG. 8. Also, by using the transmitting antenna array 24 as shown in FIG. 8 and controlling the direction in which each transmitted wave is transmitted at each timing of transmitting the transmitted wave (transmission signal), a beamformer for the direction of direction 2 shown in FIG. 8 can be realized. Further, by using the receiving antenna array 31 as shown in FIG. 8, it is possible to estimate the arrival direction of the reflected wave for the direction of direction 1 shown in FIG. 8. In this way, it becomes possible to estimate the arrival direction of the reflected wave for two substantially orthogonal angles. Therefore, it becomes possible to three-dimensionally acquire a point group indicating an object such as the subject 200.

[0083] Next, a method for detecting the heartbeat of the subject 200 by the electronic device 1 according to an embodiment will be described.

[0084] The electronic device 1 according to an embodiment transmits a transmitted wave such as a millimeter-wave radar to the subject 200, and measures (estimates) the heartbeat of the subject 200 based on the result of receiving the reflected wave reflected from the chest where the heart of the subject 200 exists. As described above, the subject 200 may be a human or an animal. In this case, for example, by frequency-filtering the vibration of the position of the subject 200 detected by the radar, a component assumed to be the envelope of the heartbeat can be extracted. Then, when the component assumed to be the envelope of the heartbeat is extracted, the approximate heartbeat interval can be calculated by taking the interval between the peaks of the envelope as the heartbeat interval. Here, an approximation that the peak of the heartbeat envelope approximately coincides with the R peak of the electrocardiogram can be used. Therefore, the "heartbeat interval" is also denoted as the RR interval or RRI (RR interval), similar to the term used in an electrocardiogram or the like.

[0085] Here, a method for estimating the heart rate interval of subject 200 will be examined based on the results of the 2D-FFT, CFAR processing, and direction-of-arrival estimation described above. First, as a result of the 2D-FFT performed in FIGS. 4 and 5, the manifestation modes of human heartbeats or body movements will be described.

[0086] FIG. 9 is a diagram showing an example of the result of receiving the reflected wave of the transmitted wave transmitted to subject 200 and performing 2D-FFT processing. FIG. 9 shows a spectrum indicating the heart sound and body movement of subject 200 as a result of the 2D-FFT. In FIG. 9, the horizontal axis represents distance (Range), and the vertical axis represents velocity (Velocity). The signal processing unit 10 (for example, the heartbeat extraction unit 13) of the electronic device 1 according to an embodiment may extract, for example, a peak Hm as shown in FIG. 9 as a body movement such as the heartbeat of subject 200. Here, the spectral components indicated by the peak Hm in FIG. 9 include not only the heart sound and the envelope of the heart sound of subject 200 but also body movements. In order to extract the heart rate interval, it is necessary to extract body movements and the like, so it is assumed that frequency filtering is performed, for example. The frequency filtering performed here is assumed to be, for example, a band-pass filter, a high-pass filter, and / or a low-pass filter targeting 0.5 Hz or more and 10 Hz or less.

[0087] FIG. 10 is a diagram for explaining a method of detecting a peak based on the envelope waveform of the heart sound obtained by the above-described frequency filtering. The graph shown in FIG. 10 shows an example of the time change of the envelope waveform of the heart sound extracted by the above-described frequency filtering. The envelope waveform shown in FIG. 10 includes a large number of peaks as shown. On the other hand, it is known that the heartbeat of subject 200 falls within a range of about 50 to 130 per minute. Therefore, by selecting peaks with an interval of 0.4 seconds to 0.8 seconds, which is the reciprocal of the heartbeat rate per minute, from among the large number of peaks shown in FIG. 10, the approximate heart sound interval of subject 200 can be calculated. For example, the peak indicated by the downward arrow in FIG. 10 may be selected as the approximate heart sound interval of subject 200.

[0088] FIG. 11 is a flowchart showing an example of the operation of estimating the heartbeat interval described above. Hereinafter, with reference to FIG. 11, the operation of estimating the heartbeat interval described above will be outlined.

[0089] FIG. 11 shows the operation of the electronic device 1 according to an embodiment after receiving the reflected wave. That is, as a premise of the operation shown in FIG. 11, the electronic device 1 shown in FIG. 2 transmits a transmission wave (transmission signal) from the transmission antenna array 24. Then, at least a part of the transmission wave transmitted from the electronic device 1 is reflected by the subject 200 (for example, the chest) to become a reflected wave. Then, the electronic device 1 shown in FIG. 2 receives such a reflected wave from the reception antenna array 31. Then, the operation shown in FIG. 11 starts.

[0090] When the operation shown in FIG. 11 starts, first, in step S110, the signal processing unit 10 of the electronic device 1 processes the received signal (received signal). The signal processing performed in step S110 may include, for example, the above-described 2D-FFT, CFAR processing, and / or direction-of-arrival estimation. Such an operation may be performed, for example, by the received signal processing unit 12 of the signal processing unit 10.

[0091] Next, in step S120, the signal processing unit 10 extracts the vibration source from the information processed in step S110. The signal processing performed in step S120 may include, for example, a process of filtering the data obtained as a result of performing the 2D-FFT process. In step S120, the signal processing unit 10 may extract the spectral components only at the position where the subject 200 exists. Here, the position where the subject 200 exists may be specified by various known methods. Such an operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0092] Next, in step S130, the signal processing unit 10 converts the processing result of the previous stage into a vibration waveform. The processing performed in step S130 may include, for example, a process of extracting phase information from IQ data. In step S130, the signal processing unit 10 may include a process of extracting vibration data including heart sounds from the spectral components of the 2D-FFT processing of the subject 200 extracted in step S120. Such an operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0093] Next, in step S140, the signal processing unit 10 extracts vibration data from the processing result of the previous stage. The processing performed in step S140 may include, for example, frequency filtering. In step S140, the signal processing unit 10 may extract a low-frequency signal including the envelope of the heart sound of the subject 200 by performing frequency filtering. Such an operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.

[0094] Next, in step S150, the signal processing unit 10 detects the peak of the pulsation of the subject 200 from the processing result of the previous stage, and calculates the RR interval (RRI) of the subject 200 by calculating the interval between the peaks. In step S150, the signal processing unit 10 may detect the peak of the low-frequency signal including the envelope of the heart sound of the subject 200. Also, in step S150, the signal processing unit 10 may calculate and / or extract the interval (interval) between each peak. Such an operation may be performed, for example, by the calculation unit 14 of the signal processing unit 10. As described above, in step S150, the signal processing unit 10 may extract the interval between heart sounds.

[0095] Next, in step S160, the signal processing unit 10 may calculate the heart rate variability (HRV) of the subject 200. In step S160, the signal processing unit 10 may calculate the HRV of the subject 200 by calculating the spectral density of the time series data of the RRI. When calculating the power spectral density of the time series data of the RRI, frequency analysis of the waveform of the time series of the RRI may be performed by using, for example, Welch's method or the like. Such an operation may be performed by, for example, the calculation unit 14 of the signal processing unit 10. Also, in step S160, the signal processing unit 10 may analyze the spectrum from the previous processing result.

[0096] As described above, the electronic device 1 according to one embodiment extracts a component that seems to be the envelope of the heartbeat by frequency filtering the vibration at the position where the subject 200 is present, and sets the interval between the peaks of the envelope as the heartbeat interval. In this way, the electronic device 1 according to one embodiment can calculate the approximate heartbeat interval of the subject 200.

[0097] In the calculation of the heartbeat interval as described above, for example, as shown in FIG. 10, several peaks exist during a time of several 10 ms. Therefore, a certain degree of uncertainty is included when selecting the peak of the heartbeat. For this reason, the accuracy of the calculated heartbeat interval also has an error of about several 10 ms. For example, in the calculation of the heartbeat interval described above, when compared with the instantaneous RRI acquired by an electrocardiograph, an error of at least about 20 ms occurs. With this accuracy, it is difficult to calculate the heart rate variability (HRV) and perform human autonomic nerve and / or emotion analysis. That is, in the calculation of the heartbeat interval described above, it is difficult to perform more advanced medical analysis by extracting heart sounds.

[0098] Therefore, for example, it is also conceivable to further cut the peak on the high-frequency side by frequency filtering so that the time interval of the heartbeat is about 0.4 seconds to 0.8 seconds. However, since the accuracy of the signal information shown in FIG. 10 is not exceeded, it seems difficult to reduce the error of about several 10 ms.

[0099] Also, in the calculation of the above-mentioned heartbeat interval, heart sounds are not extracted. Therefore, in the above-mentioned method, it seems difficult to obtain information contributing to the diagnosis based on the acoustic properties of the heart sound itself (for example, auscultation in medical treatment, etc.).

[0100] Therefore, the electronic device 1 according to one embodiment further improves the above-mentioned method. As a result, the electronic device 1 according to one embodiment analyzes the heart sound itself and extracts an accurate heartbeat interval by extracting the heart sound using, for example, a radar in a high-frequency band of millimeter waves or higher. Hereinafter, such a method will be described.

[0101] The electronic device 1 according to one embodiment narrows down the subspace and subspace basis where the signal component of the heart sound of the subject 200 exists by performing appropriate signal processing on the signal (chirp signal) received by the electronic device 1 in an appropriate order in order to extract the heart sound of the subject 200 with high accuracy. The electronic device 1 according to one embodiment may adopt different basis vectors in the linear space where the signal of the heart sound of the subject 200 exists, describe the space using an appropriate coordinate system, and extract the subspace based on each coordinate. By such processing, the electronic device 1 according to one embodiment can search for the subspace where the heart sound of the subject 200 exists. In one embodiment, such a coordinate system may be appropriately selected according to the purpose. For example, a coordinate system of a space subjected to 2D-FFT processing, a coordinate system of a time-series signal obtained by time reduction of a chirp signal, a coordinate system obtained by its Fourier transform, or a coordinate system by continuous / discrete wavelet may be used.

[0102] The electronic device 1 according to one embodiment may execute the following characteristic processing on the received signal according to a step-by-step procedure. Hereinafter, an outline of the characteristic processing by the electronic device 1 according to one embodiment is shown. The electronic device 1 according to one embodiment executes three characteristic processes. That is, the electronic device 1 according to one embodiment executes (1) extraction of a partial space, (2) generation of an envelope and selection of the most conscience sound, and (3) statistical signal processing for calculating an inter-beat interval. Hereinafter, each process will be described in more detail.

[0103] (1) Extraction of a partial space First, the electronic device 1 according to one embodiment executes a process of omitting data considered unnecessary in the linear space for the received signal. The electronic device 1 according to one embodiment may execute a process of leaving the necessary partial space by omitting the unnecessary partial space. Specifically, the following process may be executed.

[0104] (1-1) First stage: Dimension reduction In the first stage, an appropriate window function is applied to the 2D-FFT processing of the chirp signal received by the electronic device 1, and only the micro-Doppler component of the point group where the subject 200 (for example, a person or an animal) exists is extracted by arrival direction estimation. Here, a plurality of window functions centered on a plurality of distance ranges are generated in the vicinity of the approximate position of the target, and the plurality of window functions are applied. Hereinafter, the process of applying a plurality of window functions in this way is referred to as "multi-window processing". Here, one vibration time-series signal may be generated for one window function, that is, one center distance range.

[0105] (1-2) Second stage: Dimension reduction of the partial space In the second stage, principal component analysis and / or singular value decomposition of the set of time-series waveforms of a plurality of vibrations extracted in the first-stage processing are executed on the micro-Doppler signal.

[0106] (1-3) Third stage: Dimension reduction of the partial space In the third stage, frequency filtering is performed by using at least one of a short-time Fourier transform, a continuous wavelet transform, and a band-pass filter.

[0107] (1-4) Fourth stage: Reduction of the dimension of the subspace In the fourth stage, appropriate wavelet functions and scaling functions for heart sounds are used to perform multi-resolution analysis by discrete wavelet transform, thereby extracting heart sounds.

[0108] (2) Generation of an envelope and selection of the best heart sound Next, in order to facilitate processing of the extracted heart sound data, the electronic device 1 according to one embodiment may execute a process of generating an envelope from the waveform of the extracted heart sound. Further, the electronic device 1 according to one embodiment may execute a process of selecting (extracting) the best one from the data of a plurality of heart sounds. Specifically, the following processes may be executed.

[0109] (2-1) Generation of an envelope Here, an envelope is generated from the heart sound extracted in the above (1) extraction of the subspace. For the envelope generation process, for example, any one of a continuous wavelet transform, a discrete wavelet transform, a wavelet scattering coefficient, a mel-frequency cepstral coefficient, or a moving variance may be used.

[0110] (2-2) Selection of the best heart sound Next, select the best heart sound from the generated envelopes. The heart sounds generated by the extraction of the above (1) subspace exist in the number of window functions. Therefore, here, the optimal heart sound is extracted (selected) from the heart sounds that exist in the number of window functions. First, the score of the heart sound may be calculated using a pre-trained learning classifier. In this case, the learning classifier may be, for example, an autoencoder, LSTM (Long Short Term Memory), other neural networks, a support vector machine, or a decision tree. Then, based on the calculated score, the best heart sound is determined. At the same time, the accompanying calculation results may be obtained from the learning classifier. Also, as the learning classifier, any one of a linear support vector machine, a simple perceptron, one using logistic regression, a decision tree (classification tree), the k-nearest neighbor method, a random forest, a non-linear support vector machine, or a neural network, or a combination thereof may be used.

[0111] (3) Statistical signal processing for calculating the inter-beat interval Here, the inter-beat interval is calculated by performing statistical processing on the envelope of the generated best heart sound. In most cases, including healthy subjects, heart sounds include the first heart sound (S1) and the second heart sound (S2). When detecting the time interval between the peaks of heart sounds within a certain time, the following two intervals exist. (i) Between the first heart sound and the second heart sound (hereinafter, also referred to as "S1 - S2 interval") (ii) Between the second heart sound and the first heart sound (hereinafter, also referred to as "S2 - S1 interval") The distribution of the histograms or probability densities of these intervals becomes a mixture Gaussian distribution or a single Gaussian distribution. Therefore, based on information such as the distribution, an accurate inter-beat interval can be calculated, for example, by using simple DP (Dynamic Programming) matching (dynamic programming method).

[0112] According to the electronic device 1 according to one embodiment, by obtaining the envelope waveform of the heart sound obtained through the processing of each of the above-described steps, detecting the positive and negative peaks thereof, and appropriately calculating the interval therebetween, the RR interval (RRI / RR interval) can be calculated with high accuracy.

[0113] Next, the operation of the electronic device 1 according to one embodiment will be described in more detail.

[0114] FIG. 12 is a flowchart showing an example of the operation performed by the electronic device 1 according to one embodiment. FIG. 13 is a flowchart showing in more detail an example of the operation in step S16 in FIG. 12. Hereinafter, with reference to FIGS. 12 and 13, the operation flow performed by the electronic device 1 according to one embodiment will be described.

[0115] Step S11 shown in FIG. 12 can be performed in the same manner as the operation in step S110 shown in FIG. 11. That is, when the operation shown in FIG. 12 starts, first, in step S11, the signal processing unit 10 of the electronic device 1 processes the received signal (received signal). The signal processing performed in step S11 may include, for example, the above-described 2D-FFT, CFAR processing, and / or direction-of-arrival estimation. Such an operation may be performed, for example, by the received signal processing unit 12 of the signal processing unit 10.

[0116] In step S12, the signal processing unit 10 performs multi-window processing. In step S12, the signal processing unit 10 may perform a process of determining candidates for clusters of humans or animals and / or filtering processing (multi-window processing) on the data subjected to 2D-FFT processing using a plurality of window functions.

[0117] In step S12, the signal processing unit 10 may perform multi-window processing for generating a plurality of heart sound candidates as a pre-step of selecting the best heart sound candidate. Here, the signal processing unit 10 may perform a process of grouping point clouds corresponding to humans or animals as clusters on the 2D-FFT processed data. Further, the signal processing unit 10 may perform multi-window processing by applying a plurality of window functions to the selected clusters. The multi-window processing performed here may include a process of generating 2D-FFT processed data that is the basis for a plurality of heart sound candidates.

[0118] In step S12, the signal processing unit 10 may generate a plurality of candidate heart sounds by extracting only the region where the subject 200 exists on the range-Doppler plane calculated by 2D-FFT using an appropriate window function. In step S12, the signal processing unit 10 may provide a plurality of window functions as appropriate window functions. Here, the signal processing unit 10 may use, for example, window functions such as a Hanning window, a Hamming window, and a Blackman-Harris window.

[0119] In step S12, the signal processing unit 10 may detect a vibration source (target) including body movements including the heart sound and / or respiration of the subject 200, for example, based on the following first to third procedures.

[0120] (First procedure) The signal processing unit 10 classifies point clouds exceeding the CFAR threshold on the range-Doppler plane into a predetermined angular area based on the result of arrival direction estimation (see FIGS. 5 and 6). For example, assuming that the angle on the xy plane as shown in FIG. 6 is θ, it may be classified into angular areas A to C as follows. Area A: -10deg. < θ < 10deg. Area B: -20deg. < θ ≤ -10deg. Area C: 10deg. ≤ θ < 20deg.

[0121] (Second procedure) The signal processing unit 10 applies clustering to the point groups that exceed the CFAR threshold on the range-Doppler plane as shown in S1 of FIG. 5 within a group of areas of a certain angle classified in the first procedure. Here, as the clustering, a method such as DBSCAN may be applied.

[0122] (Third procedure) Assuming that L clusters have been processed in the second procedure, for the l-th cluster, the signal processing unit 10 calculates the deviation D dev [l] of the number of bins in the Doppler direction and compares it with a certain threshold D dev,th . As a result of this comparison, the signal processing unit 10 determines only those for which D dev [l] ≥ D dev,th as the subject 200 and sets the flag HF[l]. That is, the signal processing unit 10 may perform processing as shown in, for example, the following [Pseudo-code 1].

[0123] [Pseudo-code 1] for l = 1 to L do if D dev [l] ≥ D dev,th HF[l] = 1 else HF[l] = 0 end if end do

[0124] In step S13, the signal processing unit 10 restores the set of chirp signals. In step S13, the signal processing unit 10 may restore the set of chirp signals by performing an inverse discrete Fourier transform (2D-IFFT (inverse fast Fourier transform)) on the plurality of generated 2D-FFT data.

[0125] FIG. 14 is a diagram for explaining multi-window processing by the electronic device 1 according to an embodiment. FIG. 14 explains multi-window processing for generating a plurality of time-series signals by window functions moved in a plurality of range directions from the point group belonging to the cluster selected by the above-described pseudo code 1.

[0126] In the upper part of FIG. 14, the horizontal axis direction indicates distance (Range), and the vertical axis direction indicates Doppler velocity. The upper part of FIG. 14 shows the result of clustering the point group of 2D-FFT data at a certain time. The upper part of FIG. 14 shows a state in which three clusters, cluster 1, cluster 2, and cluster 3, are generated as a result of clustering. The middle part of FIG. 14 conceptually shows a state in which the signal processing unit 10 executes multi-window processing for each of cluster 1, cluster 2, and cluster 3. The middle part of FIG. 14 shows a state in which the signal processing unit 10 performs multi-window processing by applying a window function while shifting it little by little in the distance (Range) direction for each cluster. Further, the lower part of FIG. 14 shows the result of calculating the vibration velocity by performing 2D-IFFT for each of cluster 1, cluster 2, and cluster 3. The lower part of FIG. 14 shows a state in which a time-series signal of vibration is generated by performing 2D-IFFT for each cluster. Actually, a plurality of time-series signals of vibration can be obtained by applying a plurality of window functions while shifting them little by little in the distance (Range) direction for each cluster shown in FIG. 14.

[0127] In step S13 of FIG. 11, the signal processing unit 10 applies a plurality of window functions to the cluster composed of the point group selected as flag HF[l]=1 by the above-described [pseudo code 1]. By this processing, 2D-FFT data corresponding to each window function is generated. Here, assuming that the 2D-FFT data generated by the l-th window function is expressed as in the following formula (1).

Equation

[0128] In this case, in step S13, the signal processing unit 10 can calculate the signal waveform of the time series of the vibration speed according to the following formula (2). By performing two-dimensional inverse fast Fourier transform (2D-IFFT), it is restored to a set of chirp signals.

Equation

[0129] Hereinafter, as an example, only one cluster, for example, only cluster 1 shown in FIG. 14 will be described.

[0130] In step S14, the signal processing unit 10 performs analysis of the main components of the signal and / or removal of noise. Step S14 may be singular value decomposition (SVD) for preprocessing and removing the noise of the signal. This process may be performed for the purpose of removing noise with low energy and / or low-level noise mixed due to the uncertainty of the Fourier transform.

[0131] In step S14, the signal processing unit 10 may perform singular value decomposition on the set S l of the received chirp signals extracted in the 2D-FFT plane. Denoting the matrix formed by arranging the left singular vectors as U, the matrix formed by arranging the singular values diagonally as Σ, and the matrix formed by arranging the right singular vectors as V, the following formula (3) is obtained by the singular value decomposition of the set S l of the received chirp signals.

Equation

[0132] Next, the signal processing unit 10 may limit the number of row vectors in the row of the left singular vector of the above formula (3) (U ext ). Further, the signal processing unit 10 may limit the number of diagonal elements of the diagonal matrix of singular values in the above formula (3) (Σ ext ). As a result, unnecessary noise signals and vibration components other than the desired position are removed. As a result, S l The signal S projected onto the subspace of the target signal of l ext is represented by the following formula (4).

Equation

[0133] FIG. 15 is a diagram illustrating the relationship between the ranks of the target signal and the noise signal in singular value decomposition. FIG. 15 is a diagram in which the results of SVD are arranged with the horizontal axis representing the rank and the vertical axis representing the singular value in descending order of the singular values (corresponding to energy). In the present disclosure, the concept is adopted that taking out signals with a certain singular value or higher results in the target signal. In this case, the rank of the singular value that serves as the boundary between whether it is the target signal or not is the part of the rank region of 30 or less indicated by the dark gray part in the graph of FIG. 15. As shown in FIG. 15, the part of the rank region of 30 or less becomes the region of the target signal. In FIG. 15, the rank part of this target signal is set to be 1 or more and 30 or less. Therefore, in the present disclosure, for example, the singular vectors corresponding to the left and right singular values in the rank region of 1 or more and 30 or less may be taken out.

[0134] In the above example, the rank part of the target signal is set to be 1 or more and 30 or less. The maximum rank may basically be determined empirically. Also, the maximum rank (rank 30 in the present disclosure) may be determined by a statistical method. In the graph of FIG. 15, the singular value decreases rapidly at rank 130, and the signals of ranks after that basically become noise. Therefore, the subsequent rank signals may be considered unnecessary.

[0135] In the graph of FIG. 15, the left and right singular vectors (vectors that span the signal space) corresponding to the singular values in the area of ranks 31 to 130 also have some target signal components. These left and right singular vectors are assumed to be mainly caused by unnecessary fine vibrations and / or artificial noise (artifacts) generated by the radar signal processing. Therefore, such elements may be discarded.

[0136] When the noise is removed by SVD in step S14, the signal processing unit 10 converts the result of the removed noise into a signal waveform (step S15). In step S15, the signal processing unit 10 may convert the beat signal (IQ data) into a signal of the vibration time series.

[0137] In step S15, the signal processing unit 10 obtains the vector shown in the following equation (5) by taking the sum within the column vector of each chirp signal based on the above equation (4).

Equation

[0138] In step S15, finally, by taking the argument and derivative on the Gaussian plane of s l ext,sum a vector representing the vibration speed as shown in the following equation (6) is calculated.

Equation

[0139] Next, in step S16, the signal processing unit 10 performs extraction of heart sounds and / or analysis of the R-R interval on the obtained plurality of vibration waveforms. Specifically, in step S16, the signal processing unit 10 extracts heart sounds from the vibration waveforms shown in the above formula (6). In step S16, the signal processing unit 10 may execute a process of generating a waveform of the heart sound, a process of calculating the R-R interval, and / or a process of calculating heart rate variability (HRV). Thereby, the signal processing unit 10 can calculate the R-R interval. This process may be executed on the waveforms of a plurality of vibration velocities corresponding to each window function.

[0140] As described above, the process of step S16 shown in FIG. 12 may include at least a part of the processes of steps S21 to S27 shown in FIG. 13 in more detail. Hereinafter, each step of steps S21 to S27 shown in FIG. 13 will be described in more detail.

[0141] First, in step S21 shown in FIG. 13, the signal processing unit 10 performs a process (denoising) of removing noise from the time-series waveform. Here, in the process of removing noise, an empirical Bayes method or a wavelet method such as a continuous wavelet transform (CWT) may be used.

[0142] More specifically, in step S21, the signal processing unit 10 performs preprocessing for further removing unnecessary noise on the vibration velocity vector v corresponding to the l-th window function. l vib

[0143] In step S21, the signal processing unit 10 may perform noise removal processing by an empirical Bayes method and / or band limitation by continuous wavelet. Also, in step S21, the signal processing unit 10 may perform noise removal processing by frequency subtraction using a noise profile for artifact noise and the like associated with the non-linear processing from step S11 to step S14 shown in FIG. 12.

[0144] Next, in step S22, the signal processing unit 10 extracts the waveform of the target signal, that is, the heart sound waveform. In this case, for example, a method using a discrete wavelet such as a maximum overlap discrete wavelet transform (MODWT) may be used.

[0145] More specifically, in step S22, the signal processing unit 10 may perform multi-resolution analysis by discrete wavelet transform by adopting a wavelet waveform having a waveform similar to the heart sound waveform for the signal denoised as pre-processing. In this way, in step S22, the signal processing unit 10 extracts only the subspace at the level of multi-resolution analysis where the heart sound of the subject 200 empirically exists. In this way, in step S22, the signal processing unit 10 may extract the heart sound of the subject 200. Specifically, the signal processing unit 10 may use a maximum overlap multi-resolution analysis (MODWT) or the like to improve the time resolution. Also, the wavelet basis appropriate for heart beat extraction may be, for example, Symlet and Daubechies. Also, these orders may be appropriately set each time.

[0146] The signal obtained by performing the denoising process of step S21 on the signal velocity vector v l vib is v l vib,dn and further the real part Re(v l vib,dnFor (0), multi - resolution analysis shown in FIG. 16 may be performed. FIG. 16 is a diagram conceptually showing multi - resolution analysis by discrete wavelet transform. And the signal processing unit 10 can obtain the heart sound waveform h by reconstructing while empirically limiting the appropriate level to j0 ∈ N in the following formula (7).

Number

[0147] In this way, the electronic device 1 according to one embodiment can obtain a sound waveform h such as the waveforms shown in FIGS. 26 and 27 described later. This result can be directly utilized for diagnosis and treatment of heart sounds and the like.

[0148] Next, in step S23, the signal processing unit 10 generates an envelope waveform. In step S23, the signal processing unit 10 performs a scalogram by continuous wavelet transform and envelope waveform extraction by one - dimensionalization thereof in order to detect (extract) the peak of the energy of the target signal (heart sound waveform) extracted in step S22. Here, discrete / continuous wavelet transform, moving variance, and / or Hilbert transform etc. may be used.

[0149] More specifically, in step S23, the signal processing unit 10 may obtain a scalogram by continuous wavelet transform in order to obtain the envelope waveform of the heart sound waveform h shown in the above formula (7). The signal processing unit 10 takes the sum for a certain range f1 to f2 on the frequency axis at each time of the scalogram. Thereby, as shown in FIGS. 17 and 18, the signal processing unit 10 obtains a one - dimensional waveform s l l h ​To obtain. FIGS. 17 and 18 are diagrams showing examples of analysis results of continuous wavelet transform. This analysis can be replaced with discrete wavelet transform with sufficient attention to resolution.

[0150] FIG. 17 shows the scalogram of the heart sound waveform h l . FIG. 17 is a diagram showing the time change of the normalized vibration frequency. The horizontal axis of FIG. 17 indicates time in units of the number of samples, and the vertical axis of FIG. 17 indicates the normalized vibration frequency for each time.

[0151] Also, FIG. 18 is a diagram showing the result of one-dimensional processing of the scalogram of the heart sound waveform h l shown in FIG. 17 by continuous wavelet transform. FIG. 18 is a diagram showing the time change of the vibration speed. The horizontal axis of FIG. 18 indicates time in units of the number of samples, and the vertical axis of FIG. 18 indicates the vibration speed.

[0152] Here, the matrix representing the absolute value of the scalogram is assumed to satisfy the following equation (8). [Equation] In this case, the one-dimensional envelope waveform e l h is represented as shown in the graph of FIG. 18. The signal processing unit 10 can calculate this envelope waveform e l h by the following [Pseudo Code 2].

[0153] [Pseudo Code 2] for m = 1:M [Equation] end Here, e l h represents the vector of the one-dimensional envelope waveform, and f represents each frequency of the scalogram.

[0154] Next, in step S24, the signal processing unit 10 selects the best heart sound. In step S24, the signal processing unit 10 may perform discrimination processing by machine learning in order to extract the best heart sound envelope from the envelope waveforms of the plurality of heart sounds extracted in step S23. Here, machine learning such as an autoencoder, LSTM (Long Short Term Memory), or other neural network, or a support vector machine (SVM) may be used.

[0155] More specifically, in step S24, the signal processing unit 10 selects the optimal one from the vectors e obtained as the envelope waveforms of l heart sounds. l h Here, in order for the signal processing unit 10 to select the optimal e l h it may select the one with the smallest cost function by scoring each cost function. Here, the cost function is calculated using the reconstruction error by an autoencoder (self-encoder).

[0156] FIG. 19 and FIGS. 20(A) and 20(B) are diagrams conceptually explaining the process of classifying and discriminating the envelope of the best heart sound. FIG. 19 is a diagram conceptually showing a state in which a large number of learning data are learned by an autoencoder. FIGS. 20(A) and 20(B) are diagrams showing an example of verifying the error when data is reconstructed by an autoencoder. The vertical axis in FIGS. 20(A) and 20(B) represents an arbitrary unit determined by the digital scale of signal processing.

[0157] In step S24, the signal processing unit 10 may perform the discrimination process of the heart sound envelope for selecting the best heart sound as follows in sub-steps 241 to 243.

[0158] <Sub-step S241> In sub-step S241, the signal processing unit 10 may prepare a large number of learning data for a data set corresponding to the envelopes of the first sound and the second sound among the heart sound envelopes shown in the upper part of FIG. 19, and train the autoencoder. Here, let the number of vector dimensions of each of the learning data be denoted as J. The upper part of FIG. 19 shows data augmentation and automatically extracting appropriate heart sound envelope monophones. In this way, by training the learning data, a trained autoencoder is generated as shown in the lower part of FIG. 19.

[0159] <Sub-step S242> In sub-step S242, as shown in FIGS. 20(A) and 20(B), the signal processing unit 10 may perform the reconstruction of the trained autoencoder. In this case, the signal processing unit 10 may perform the reconstruction using the i-th vector e l h sequentially cut out by the length of J from the envelope vector e l,i h as the object to be classified and identified. Next, the signal processing unit 10 calculates the mean absolute percentage error (MAPE) between the original envelope waveform vector e l,i h and the reconstructed envelope waveform vector shown in the following formula (10).

Equation

[0160] In this case, the signal processing unit 10 may calculate the MAPE while sequentially moving the window function for calculating the cost function as shown in the following [Pseudo Code 3].

[0161] [Pseudo Code 3]

Equation

Equation

[0162] However, length is a function representing the length of the signal vector, and trunc is a function for rounding by truncating the decimal part. In the above [Pseudo-code 3], for e l h The last part that does not reach the length J is processed by looping so as not to include it in the calculation by truncating.

[0163] In FIGS. 20(A) and 20(B), one of the solid-line graph and the broken-line graph may show a waveform based on the detected data, and the other of the solid-line graph and the broken-line graph may show a waveform based on the reconstructed data. FIG. 20(A) shows data where MAPE exceeds the threshold value, that is, data for which prediction cannot be made well (cannot be reconstructed appropriately), and shows a waveform that is not relatively good. On the other hand, FIG. 20(B) shows data where MAPE does not exceed the threshold value, that is, data for which prediction can be made well (can be reconstructed appropriately), and shows a waveform that can be said to be relatively good.

[0164] <Sub-step S243> FIGS. 21(A) and 21(B) are diagrams showing examples of MAPE. FIGS. 21(A) and 21(B) are diagrams for explaining a process of extracting a minimum point of MAPE in the result of reconstruction by an autoencoder in order to select the most conscience heart sound. The horizontal axis of FIGS. 21(A) and 21(B) indicates time in units of the number of samples, and the vertical axis of FIGS. 21(A) and 21(B) indicates MAPE (unit: %). FIG. 21(A) shows an example of MAPE of the most conscience heart sound envelope l env An example is shown. Further, FIG. 21(B) shows an example of MAPE of a heart sound envelope that is not the best l env An example is shown.

[0165] In the electronic device 1 according to one embodiment, the signal processing unit 10 may provide a certain threshold MAPE in the MAPE as shown in FIG. 21(A) or FIG. 21(B). Then, the signal processing unit 10 uses the threshold MAPE th th ​A cost function may be calculated by a minimum point that is lower than [a certain value]. For example, the dashed lines shown below the graphs in FIGS. 21(A) and 21(B) respectively indicate the threshold lines when MAPE th = 10 as an example. Here, as the cost function, the number of minimum points below the threshold is used. When such a cost function is expressed by a mathematical formula, it becomes as shown in the following formula (13). [Number]

[0166] In the above formula (13), LocalMin is a function that lists minimum points. In the electronic device 1 according to one embodiment, the signal processing unit 10 may select, as the best heart sound, the one for which the cost function shown in the above formula (13) is minimized. That is, the signal processing unit 10 may select the optimal heart sound number l best as shown in the following formula (14). [Number]

[0167] The MAPE shown in FIG. 21(A) l env has a relatively large number of minimum points below the threshold MAPE th (= 10%). Therefore, the signal processing unit 10 may use the envelope shown in FIG. 21(A) as an example of the envelope of the best heart sound. Also, the MAPE shown in FIG. 21(B) l env has relatively few minimum point values below the threshold MAPE th (= 10%). Therefore, the signal processing unit 10 may use the envelope shown in FIG. 21(B) as an example of the envelope of a heart sound that is not the best.

[0168] Next, in step S25, the signal processing unit 10 statistically analyzes the best heart sound. In step S25, the signal processing unit 10 may extract the positive peak and negative peak of the envelope of the best heart sound selected in step S24 and perform statistical analysis between the peaks. Thereby, information for calculating the RRI is generated. In step S25, the signal processing unit 10 may use the estimation of the distribution of the Gaussian Mixture Model (GMM), or the EM algorithm (expectation-maximization algorithm) or the variational Bayes method.

[0169] Hereinafter, with reference to FIGS. 22 and 23, a process of generating a frequency distribution (histogram) of the peak intervals of the heart sound envelope will be described. FIG. 22 is a diagram showing an enlarged example of the envelopes of two heart sounds. The horizontal axis of FIG. 22 indicates time in units of the number of samples, and the vertical axis of FIG. 22 indicates the signal level. FIG. 23 is a diagram showing an example of a histogram of peak intervals. The horizontal axis of FIG. 23 indicates the time between peaks, and the vertical axis of FIG. 23 indicates the frequency.

[0170] FIG. 22 shows an example of the envelopes of the first heart sound (S1) and the second heart sound (S2). Also, in FIG. 22, the time interval between S1 and S2 (between S1 - S2) and the time interval between S2 and S1 (between S2 - S1) are shown. FIG. 23 shows the frequency distributions between S1 - S2 and between S2 - S1. FIG. 23 shows both histograms corresponding to between S1 - S2 and between S2 - S1, respectively.

[0171] The probability density distribution obtained by normalizing this histogram is a mixture Gaussian distribution of two clusters. Therefore, based on this information, in order to estimate the RRI later, it is necessary to estimate the mixture Gaussian distribution. Here, the mixture Gaussian distribution is a distribution described by the linear sum of a plurality of Gaussian distributions, as shown in the following equations (15) and (16). As shown in the histogram of peak intervals in FIG. 23, the probability distribution between the peaks of the heart sound is this mixture Gaussian distribution.

Equation

[0172] Hereinafter, the process of estimating the mixture Gaussian distribution using the variational Bayes method will be described. This process may include the following two processes: [First Process] and [Second Process]. Hereinafter, [First Process] and [Second Process] will be described in more detail.

[0173] [First Process] The envelope of the heart sound is cut out for each time frame numbered k = 1, 2, …, K at every length T. The envelope of the heart sound (shown in the following formula (17)) is cut out. [Number] However, the cutting out of the envelope of the heart sound shown in the above formula (17) may be performed while overlapping the envelope waveforms for each time frame.

[0174] In the above [First Process], the signal processing unit 10 divides the selected heart sound (shown in the above formula (17)) into K time frames as the best heart sound. That is, the signal processing unit 10 generates the cut - out envelope waveform as shown in the following formula (18). [Number]

[0175] FIG. 24 is a diagram for explaining the process of extracting the envelope of heart sounds. In the envelope of heart sounds shown on the upper side of FIG. 24, time windows for extraction are shown for time frames t = 1 and t = 2. In the envelope of heart sounds shown on the lower side of FIG. 24, time windows for extraction are shown for time frames t = T - 1 and t = T. Here, T and t are discrete times and represent non - negative integers. Also, in the envelope of heart sounds shown on the upper side of FIG. 24, the width w win of the time window is shown, and the overlapping width w o is shown. Here, the moving step of the time window is w step = w win - w o and becomes like this.

[0176] [Second Process] Using the mixture Gaussian distribution latent variable at time t = T as the prior distribution, input it and update the latent variable at t = T + 1 by the variational Bayes method.

[0177] In the above [Second Process], the signal processing unit 10 performs Bayesian estimation for the time frame t = k using the variational Bayes method described below, including the latent parameters of the mixture Gaussian distribution.

[0178] Here, an outline of the estimation by the variational Bayes method is described. The variational Bayes method is a method for estimating the posterior distribution of the latent variable Z. Generally speaking, the variational Bayes method is a method for approximating a certain approximate posterior distribution q(Z) to the actual posterior distribution p(Z|X) by the variational method. In the variational Bayes method, the latent variable and the variable are synonymous. In the variational Bayes method, all variables are treated as latent variables whose true values cannot be calculated, that is, random variables. Hereinafter, the outline of the variational Bayes method will be explained by generalizing the latent variable.

[0179] In the variational Bayes method, instead of minimizing the Kulback-Leibler divergence (KL divergence) described by the following equation (19), an approximation is made by maximizing the following equation (20). Equation (20) is called the Evidence of Lower Bound (ELBO).

Number

Number

[0180] That is, the signal processing unit 10 can obtain the approximate posterior distribution q(Z) of the latent variable Z by solving the following equation (21).

Number

Number

[0181] Under the conditions of the above equation (22), the signal processing unit 10 may perform a process of solving the Euler-Lagrange equation shown in the following equation (23) for the above equation (19).

Number

[0182] Next, a method of applying the variational Bayes method described so far to the estimation of an actual mixture Gaussian distribution will be explained.

[0183] First, it is necessary to perform mean field approximation by grouping the latent variables of the approximate posterior distribution. To use the variational Bayes update equation (24), first, considering the dependencies of the latent variables of the mixture Gaussian distribution, the joint probability density distribution is decomposed into each probability distribution as shown in the following equation (26) using Bayes' theorem. [Mathematics] Here, the latent variables of the mixture Gaussian distribution are, for K Gaussian distributions, the mixing rate of the Gaussian distributions π = [π1, π2,..., π K , the expected value of the Gaussian distributions μ = [μ1, μ2,..., μ K , the variance of the Gaussian distributions Σ = [Σ1, Σ2,..., Σ K , and the one-hot vector indicating which Gaussian distribution the n data belong to z n = [z n1 , z n2 ,..., z nK . Considering the dependencies of these latent variables of the mixture Gaussian distribution, the joint probability density distribution is decomposed into each probability distribution as shown in the above equation (26) using Bayes' theorem. The decomposition shown in the above equation (26) may be performed after clarifying the dependencies of the latent variables using a graphical model.

[0184] The conjugate prior distributions of the probability distributions that are the factors of the above formula (26) are set as in the following formulas (27) to (31). For a certain likelihood function, when the prior distribution and the posterior distribution have the same functional form, the prior distribution and the posterior distribution are called conjugate prior distributions with respect to the likelihood function.

Number

Number

Number

Number

Number

[0185] Here, regarding formula (29), P(π|α) is a Dirichlet distribution with α = [α1, α2, …, α K as parameters. Also, B(α) is a beta function extended to multiple variables. Regarding formula (31), W(Σ k |W, ν) is a Wishart distribution of Σ k with ν, D, and W as parameters. Also, ν is the number of data vectors. D is the number of variables. W is the covariance of Σ k . C(W, ν) is a constant calculated using the gamma function as shown in the following formula (32).

Number

[0186] Next, regarding the approximate posterior distribution, it is grouped into a one-hot vector z and other variables, and formula (24) is specifically defined as in the following formula (33).

Number

[0187] Originally, different from algorithms such as the EM algorithm, the variational Bayes method is performed without distinguishing between latent variables and general variables. Therefore, in the variational Bayes method, there is no particular meaning in distinguishing between the E-step (Expectation step) and the M-step (Maximization step). However, hereinafter, for convenience, the above two steps are defined as follows. E-step: Take the expected value for Group 2 and update the approximate posterior distribution q(z) of z. M-step: Take the expected value for q(z) of Group 1 obtained in the E-step and update the approximate posterior distribution q(π, μ, Σ) of Group 2.

[0188] Hereinafter, as the signal processing performed by the signal processing unit 10 in the electronic device 1 according to an embodiment, the processing of the variational Bayes method regarding the mixture Gaussian distribution from time t = T - 1 to time t = T will be described. This processing may include five sub-steps from Sub-step 251 to Sub-step 255 with the above E-step and M-step in between.

[0189] <Sub-step S251> In Sub-step S251, each latent variable is initialized (when T = 0), or the latent variable at t = T - 1 is inherited as the prior distribution (T ≥ 1). Here, the latent variable of the posterior distribution calculated at time t = T is given as the initial value at time t = T. Here, the latent variable of the posterior distribution calculated at time t = T may be α| t=T , β| t=T-1 , ν| t=T-1 , m| t=T-1 , W| t=T-1 And the initial value at time t = T may be α0| t=T , β0| t=T , ν0| t=T , m0| t=T , W0|t=T It may be. The subscript 0 is attached as representing the initial value.

[0190] <Sub-step S252> In sub-step S251, the approximate posterior distribution q(z) of z is calculated by the E-step, and the burden rate r nk (posterior probability of one-hot vector) is calculated. Here, as the E-step, using the above formula (33), the above formula (24) is rewritten as the following formula (34). [Number]

[0191] Using the above formula (34), the burden rate r nk (posterior probability of one-hot vector) is calculated by the following formulas (35) and (39). [Number] [Number] [Number] [Number] [Number]

[0192] <Sub-step S253> In sub-step S253, as the M-step, using the burden rate r nk , the latent variable (group 2) of the mixture Gaussian distribution is updated. Here, n is the index of the data. Also, k is the index of the Gaussian distribution. Here, as the M-step, the above formula (24) is rewritten as the following formula (40). [Number]

[0193] Update the parameters using the above formula (40) and the following formulas (41) to (48).

Number

Number

Number

Number

Number

Number

Number

Number

[0194] <Sub-step S254> In sub-step S254, perform convergence determination by repeating sub-step S252 and sub-step S253, and end the processing of sub-step S252 and sub-step S253. Specifically, between the steps of the iteration, it may be determined that convergence has occurred when the increase in the log-likelihood function or the increase in the ELBO is less than or equal to the set value, or when the maximum number of iterations is exceeded.

[0195] In sub-step S254, the ELBO shown in formula (20) can be used as an index for convergence determination. When formula (20) is rewritten in accordance with the estimation of the mixture Gaussian distribution, it becomes the following formula (49).

Number

Number

[0196] <Sub-step S255> In sub-step S255, for each latent variable, by taking the expected value of each distribution as the representative value, a Gaussian mixture model (GMM) is determined. The variational Bayes method calculates the posterior distribution of the Gaussian mixture distribution. Therefore, in order to finally determine the Gaussian mixture distribution, it is necessary to determine the representative value. Here, for \(\pi\), \(\mu\), and \(\Sigma\) among each latent variable, by taking the expected value, the Gaussian mixture distribution is determined. That is, as shown in the following formulas (51) to (53), the Gaussian mixture distribution is determined with each expected value as the representative value.

Number

Number

Number

[0197] From the above, the Gaussian mixture distribution at time \(t = T\) obtained from the representative value of the approximate posterior distribution is as shown in the following formulas (54) and (55). The Gaussian mixture distribution obtained in this way may be used in the next step S26.

Number

Number

[0198] In reality, as shown in FIGS. 22 and 23, in many cases including healthy individuals, only the first heart sound (S1) and the second heart sound (S2) are prominent. In such cases, it is simplified to K = 2. For subjects with abnormalities such as abnormal heartbeats, the third heart sound and / or the fourth heart sound may become prominent. In such cases, K = 3 or 4.

[0199] As described above, [the second process] may be completed, and the process of step S25 may also be completed.

[0200] Next, in step S26, the signal processing unit 10 calculates the RR interval (RRI) based on the statistical information of the heart sound intervals generated in step S25.

[0201] More specifically, in step S26, the signal processing unit 10 may estimate the RRI using the mixture Gaussian distribution (the above equations (54) and (55)) calculated in step S25. In this case, the signal processing unit 10 first estimates the distribution of the RRI using the above equations (54) and (55).

[0202] For the reproductive property of the normal distribution, that is, for two random variables X and Y following the normal distribution, the following equation (56) holds.

Equation

[0203] Therefore, considering FIGS. 22 and 23 for the above equations (54) and (55), the distribution of the RRI is estimated as the following equation (57). However, in equation (57), μ RRI and Σ RRI are as follows in the following equation (58).

Equation

Equation

[0204] Here, in FIGS. 22 and 23, let t be the vector of the times of the peaks of the envelopes of the heart sounds picked up. peak Then, this is expressed as the following equation (59).

Equation

[0205] Next, let the vector obtained by arranging non - negative integer multiples of the mean value μ of the probability density distribution of the RRI defined by the above equations (57) and (58) be t RRI . Then, this is expressed as the following equation (60). targ

Equation

Equation

[0206] t targ is the time that serves as a target criterion for selecting candidates corresponding to the points where the RRI is taken from t peak . Hereinafter, by performing a method similar to DP (Dynamic Programming) matching between the above equations (59) and (60), the peak points of the envelopes of the heart sounds considered as the RRI are extracted.

[0207] FIG. 25 is a diagram for explaining the DP matching for RRI estimation. In FIG. 25, the vector t peak is arranged in the horizontal axis direction, and the vector t targ is arranged in the vertical axis direction. In FIG. 25, the circled marks indicate that t peak and t targ are matched. Also, in FIG. 25, the cross marks indicate the points where the matching was attempted according to the following [Pseudo - code 4] but no matching occurred. Let the vector of the flags indicating whether t peak matches any of the elements of t targ be m RRI . m RRI can be expressed as the following equation (61).

Equation

[0208] [Pseudo-code 4] [Number] for l = 1 to L (vertical axis loop) for n = 1 to N (horizontal axis loop) [Number] if D t ≤ D th [Number] (Write 1 to the matching flag) break (Once the matching flag becomes 1, break the horizontal axis loop) else [Number] (Write 0 to the matching flag) end n = n + 1 (Increment the iteration of the horizontal axis in Figure 25) end l = l + 1 (Increment the iteration of the vertical axis in Figure 25) end

[0209] Here, the above formula (62) means determining the threshold D of the distance between t RRI and t peak by multiplying the variance Σ of the probability density distribution of RRI shown in the following formula (66) by a constant a targ . th . [Number]

[0210] The above formula (63) is for t peak and t targIt means the process of calculating the distance between elements within a loop. That is, in [Pseudo-code 4], the elements of t that meet the condition shown in the following formula (67) peak are extracted as the matching points.

Number

[0211] As the calculation result of [Pseudo-code 4], let M be the number of non-zero elements of m RRI , that is, the following formula (68) holds.

Number

[0212] Next, take the Hadamard product (element-wise product) of the matching flag m RRI and t peak , and exclude 0, then the time series vector t that becomes the point of RRI RRI can be calculated. That is, the result of extracting only the points marked with black dots in the grid in Figure 25 is shown as the following formula (69).

Number

[0213] The vector Δt indicating the RRI data RRI is calculated by taking the difference between adjacent elements of t RRI as shown in the following formula (70).

Number

[0214] Finally, the vector Δt indicating the RRI data RRI requires missing value processing and smoothing processing. The matching in the above processing of [Pseudo-code 4] is in the l-th iteration in the loop of Figure 25, and in all cases, D t ≤ Dth If it does not become so, t RRI is missing, and Δt RRI In the elements of, an outlier with a very large value may occur. This missing / outlier processing can be performed by allowing up to a certain magnification b with respect to the average value of Δt shown in the following equation (71), and replacing it with the average value if it is larger. RRI can be performed by allowing up to a certain magnification b with respect to the average value of Δt shown in the following equation (71), and replacing it with the average value if it is larger.

Number

[0215] [Pseudo-code 5] for m = 1:M - 1

Number

Number

[0216] In this way, the RRI data after performing the missing / outlier processing is marked with a dash as Δt' RRI is denoted. By further applying a moving average filter or the like to this, the data Δt'' RRI smoothed as necessary can be used as the final RRI data. An example of the data of Δt'' RRI obtained as described above is shown in FIG. 28 for FIGS. 26 and 27 showing examples of heart sound waveforms. FIGS. 26, 27, and 28 are diagrams showing examples of time-series waveforms of heart sounds and RRI by the electronic device 1 according to one embodiment. In FIGS. 26 and 27, the horizontal axis represents time, and the vertical axis represents vibration velocity. Also, in FIG. 28, the horizontal axis represents time, and the vertical axis represents RRI. FIG. 26 is a diagram showing a time-series waveform of a heart sound obtained by the electronic device 1 according to one embodiment. FIG. 27 is a diagram showing an enlarged view of the time span of the region surrounded by the broken line in FIG. 26.

[0217] As shown in FIG. 27, according to the time-series waveform of the heart sound obtained by the electronic device 1 according to one embodiment, the first heart sound S1 and the second heart sound S2 can be clearly identified. Also, as shown in FIG. 27, according to the time-series waveform of the heart sound obtained by the electronic device 1 according to one embodiment, the R-R interval (RRI) calculated from the interval between the first heart sound S1 and the second heart sound S2 can also be clearly identified. Here, although the RRI and the heart rate interval are not exactly the same, they are considered to be approximately the same. Therefore, in the present disclosure, the RRI and the heart rate interval will be described as indicating the same thing.

[0218] The results shown in FIG. 28 are calculated based on the heart sounds accurately extracted in FIGS. 26 and 27. Therefore, the results shown in FIG. 28 have an accuracy of several milliseconds.

[0219] Next, in step S27, the signal processing unit 10 calculates the heart rate variability (HRV). In step S27, the signal processing unit 10 may calculate the power spectral density (PSD) by performing frequency analysis on the time-series waveform of the RRI. The frequency analysis of the time-series waveform of the RRI may use, for example, the Welch method or the like.

[0220] More specifically, in step S27, the signal processing unit 10 may perform FFT and power spectrum calculation on the Δt´´ RRI estimated in step S26 according to the Welch method or the like, that is, by a window function that overlaps in time. Thereby, the signal processing unit 10 can obtain the power spectral density (PSD) of the RRI.

[0221] FIG. 29 is a diagram showing an example of the power spectral density (PSD) of the RRI obtained by the signal processing unit 10. In FIG. 29, the horizontal axis represents frequency, and the vertical axis represents the power spectral density (PSD). As the power spectrum calculation method such as the Welch method, a known general method may be adopted. Therefore, a more detailed description of the power spectrum calculation method such as the Welch method will be omitted.

[0222] FIG. 29 is a diagram showing an example of power spectral density calculated from the time series waveform of the RRI shown in FIG. 28 using the Welch method. According to the electronic device 1 according to one embodiment, the power spectral density of the RRI can be calculated based on the RRI with an accuracy of several milliseconds. Therefore, according to the electronic device 1 according to one embodiment, the components from 0.15 Hz to 0.4 Hz, which are called HF of the heartbeat PSD, can also be accurately calculated.

[0223] As described above, according to the electronic device 1 according to one embodiment, it is possible to obtain detailed heart sound waveforms as shown in FIGS. 26 and 27, accurate RRI time series data as shown in FIG. 28, and the power spectral density (PSD) of the inter-beat interval as shown in FIG. 29. According to the electronic device 1 according to one embodiment, the heartbeat of a human body or the like can be detected with good accuracy by transmitting and receiving radio waves. Therefore, according to the electronic device 1 according to one embodiment, weak vibrations such as the heartbeat of a human body or the like can be detected with good accuracy by transmitting and receiving radio waves such as millimeter waves, and it is expected to be useful in a wide variety of fields.

[0224] Here, in the examples of FIGS. 26 to 29, only the prominent first heart sound S1 and second heart sound S2 of a healthy person are described. However, according to the electronic device 1 according to one embodiment, even when the third heart sound and / or fourth heart sound generated due to an abnormal heartbeat occur, the same processing can be performed.

[0225] (Other embodiments) Hereinafter, other embodiments will be described.

[0226] In another embodiment, the electronic device 1 according to one embodiment may calculate the heart rate from the RRI data instead of obtaining the PSD from the RRI data. In this case, the signal processing unit 10 of the electronic device 1 may execute a process of calculating the heart rate from the RRI data, for example, in step S27 shown in FIG. 13. In this case, the signal processing unit 10 may perform, for example, the following processes 1 to 3 in step S27. 1. Low-pass filter (passing frequencies of about 1 Hz or less) processing 2. Process of taking the reciprocal of the RRI 3. Process of rounding the reciprocal obtained in step 2 to a natural number Through these processes, the heart rate can be calculated from the RRI. In one embodiment, the electronic device 1 according to one embodiment may perform, for example, LPF processing, reciprocal calculation, and / or rounding to a natural number as the processing of step S27 shown in FIG. 13.

[0227] In other embodiments, when the electronic device 1 according to one embodiment performs the processing shown in FIG. 13 and the above-described processes 1 to 3, the processing shown in step S24 shown in FIG. 13 may be changed. For example, the electronic device 1 according to one embodiment may execute an autoencoder, template matching based on a cross-correlation function, classification and identification by LSTM, and / or classification and identification of other-dimensional vectors by SVM as step S24 shown in FIG. 13.

[0228] In other embodiments, in the processing as shown in FIG. 13, the electronic device 1 according to one embodiment may perform the processing of step S25 by the EM algorithm. However, the EM algorithm is not a Bayesian estimation method. Therefore, the EM algorithm does not have a mechanism for calculating the posterior distribution from the prior distribution, and when used for estimating a mixture Gaussian distribution, the variables π, μ, Σ that describe the mixture Gaussian part will be point-estimated. Therefore, when executing step S25 shown in FIG. 13 by the EM algorithm, one of the following approaches will be taken. · For each time frame, perform mixture Gaussian distribution estimation by the EM algorithm without considering the relationship between time frames. · For each time frame, after performing mixture Gaussian distribution estimation by the EM algorithm, for the variables π, μ, Σ of each mixture Gaussian distribution, a moving average before and after may be taken, or time series filtering by a Kalman filter may be performed.

[0229] In one embodiment, the transmission antenna array 24 and / or the reception antenna array 31 included in the electronic device 1 is not limited to the arrangement shown in FIG. 8. For example, in one embodiment, the reception antenna array 31 included in the electronic device 1 may have a configuration as shown in FIG. 30. FIG. 30 is a diagram showing an example of a URA (Uniform rectangular array) reception antenna. By adopting the URA reception antenna as shown in FIG. 30, the arrival direction estimation at two angles may be performed only by the URA reception antenna without changing the directivity by the beamformer in the transmission antenna array 24.

[0230] In the electronic device 1 shown in FIG. 2, the signal processing unit 10 has been described as including functional units such as a heartbeat extraction unit 13 and a calculation unit 14. However, in one embodiment, the processing performed by the heartbeat extraction unit 13 and / or the calculation unit 14 may be performed by an external computer or processor or the like.

[0231] Also, in one embodiment, the processing (singular value decomposition / principal component analysis processing) in step S14 shown in FIG. 12 may be substituted by another subspace method.

[0232] Also, in one embodiment, the processing (singular value decomposition / principal component analysis processing) in step S14 shown in FIG. 12 may be omitted when the number of samples N of one chirp can be made a large number. Also, in one embodiment, the processing (singular value decomposition / principal component analysis processing) in step S14 shown in FIG. 12 may also be omitted when other noise mixing can be excluded by a hardware method or the like.

[0233] In the processing (generation of heart sound envelope waveform) of step S23 shown in FIG. 13, it was calculated by adding the results of the continuous wavelet transform in the frequency axis direction according to [Pseudo code 2]. However, an envelope waveform may be generated by other methods, for example, moving average or Hilbert transform.

[0234] The electronic device 1 according to one embodiment may calculate the RRI by taking the interval between the negative peaks of the output of the reconstruction error of the autoencoder in the power spectral density (PSD) of the RRI shown in FIG. 29. That is, the electronic device 1 according to one embodiment may use the graph 29 showing the reconstruction error of the autoencoder instead of the heart sound envelope shown in FIG. 24, and further use the points corresponding to the RRI as the negative peaks in FIG. 29. The time resolution at that time is limited by the window movement step in [Pseudo-code 3]. To reduce the time resolution, the window movement step in [Pseudo-code 3] may be reduced.

[0235] FIGS. 31 and 32 are diagrams for explaining the negative peaks of the reconstruction error output of the autoencoder. FIG. 31 is a diagram showing the MAPE of the best heart sound envelope, similar to FIG. 21(A). The horizontal axis in FIG. 3 indicates time in units of the number of samples, and the vertical axis in FIG. 31 indicates MAPE (unit: %). FIG. 32 is a diagram showing an enlarged view of the portion surrounded by the broken line in FIG. 31. As shown in FIG. 32, a negative peak when the heart sound template matches can be obtained from the graph showing the reconstruction error of the autoencoder.

[0236] The electronic device 1 according to one embodiment may use the process in step S25 of FIG. 13 in the estimation of the mixture Gaussian distribution by the variational Bayes method. That is, the electronic device 1 according to one embodiment may use the process in step S25 of FIG. 13 only instead of the histogram of the heart sound intervals shown in FIGS. 22 and 23. In this case, the mixture Gaussian distribution is unimodal, that is, one Gaussian distribution. In this embodiment, the time interval between the negative peaks in FIGS. 31 and 32 calculated by the above method may be histogrammed and used for the calculation in step S26 shown in FIG. 13. This time interval will correspond only to the heart sound interval. The histogram of this embodiment may be considered to have a single-packet characteristic. However, in this embodiment, when including some noise, it may be assumed that the variational Bayes method separates the mixture Gaussian distribution (the time between the negative peaks in FIG. 32 and the time intervals due to other noise peaks) to obtain the time between the negative peaks in FIG. 32 that is originally desirable.

[0237] The electronic device 1 according to one embodiment may implement the process in step S25 of FIG. 13 by Markov Chain Monte Carlo methods (MCMC methods).

[0238] As described above, the electronic device 1 according to one embodiment detects a weak vibration such as a heartbeat using, for example, a millimeter-wave sensor including a plurality of transmission antennas and a plurality of reception antennas. When no target is detected, the electronic device 1 according to one embodiment detects the body movement of the target while changing the beamforming pattern of the transmission antenna by changing the transmission phase of the antenna. On the other hand, when the electronic device 1 according to one embodiment detects the body movement of the target, it performs beamforming in the direction of the body movement to detect the heartbeat. Thus, according to the electronic device 1 according to one embodiment, the signal quality can be improved by automatically detecting the direction of the human body. Therefore, according to the electronic device 1 according to one embodiment, the detection accuracy and / or the detection range of the heartbeat can be improved. For this reason, according to the electronic device 1 according to one embodiment, the human heartbeat can be detected with high accuracy.

[0239] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications or corrections based on the present disclosure. Therefore, it should be noted that these modifications or corrections are included in the scope of the present disclosure. For example, the functions included in each functional unit can be rearranged so as not to be logically contradictory. A plurality of functional units, etc. may be combined into one or divided. Each of the embodiments according to the present disclosure described above is not limited to being faithfully implemented in each of the described embodiments, and can be implemented by appropriately combining each feature or omitting a part thereof. That is, those skilled in the art can make various modifications and corrections based on the present disclosure for the content of the present disclosure. Therefore, these modifications and corrections are included in the scope of the present disclosure. For example, in each embodiment, each functional unit, each means, each step, etc. can be added to other embodiments so as not to be logically contradictory, or replaced with each functional unit, each means, each step, etc. of other embodiments. Also, in each embodiment, a plurality of each functional unit, each means, each step, etc. can be combined into one or divided. Also, each of the embodiments of the present disclosure described above is not limited to being faithfully implemented in each of the described embodiments, and can also be implemented by appropriately combining each feature or omitting a part thereof.

[0240] The above-described embodiments are not limited to being implemented only as the electronic device 1. For example, the above-described embodiments may be implemented as a control method for a device such as the electronic device 1. Further, the above-described embodiments may be implemented, for example, as a program executed by a device such as the electronic device 1, or a storage medium or recording medium on which the program is recorded.

[0241] Also, the electronic device 1 according to the above-described embodiment has been described as including components that constitute a so-called radar sensor, such as the transmission antenna array 24 and the reception antenna array 31. However, the electronic device according to one embodiment may be implemented as a configuration such as the signal processing unit 10, for example. In this case, the signal processing unit 10 may be implemented as having a function of processing signals handled by, for example, the transmission antenna array 24 and the reception antenna array 31.

Explanation of Signs

[0242] 1 Electronic device 10 Signal processing unit 11 Signal generation processing unit 12 Received signal processing unit 13 Heartbeat extraction unit 14 Calculation unit 21 Transmission DAC 22 Transmission circuit 23 Millimeter-wave transmission circuit 24 Transmission antenna array 31 Reception antenna array 32 Mixer 33 Reception circuit 34 Reception ADC 50 Communication interface 60 External device

Claims

1. A transmitting unit that transmits a transmission wave, A receiving unit that receives a reflected wave from a target of the transmission wave, A signal processing unit that detects the distance position, direction, and speed of the target based on a conversion signal obtained by Fourier-transforming the beat signal of the transmission wave and the reflected wave, An electronic device comprising: The signal processing unit includes an extraction unit that extracts heartbeat information of the target based on the detected distance, direction, and speed of the target, The extraction unit, Extracts signal components corresponding to vibrations associated with the heartbeat of the target from the conversion signal using a plurality of window functions centered on a plurality of distance ranges, Selects the most appropriate heart sound signal as a heart sound by applying a learning classifier to the extracted signal components, Selects the most appropriate optimal signal as a heartbeat by performing envelope processing on the most appropriate heart sound signal, An electronic device that obtains a value indicating the interval of peak times from the optimal signal and extracts the heartbeat interval of the target based on the variance and central value of the value indicating the interval.

2. The extraction unit, Extracts the signal components by paying attention to a plurality of positions of the conversion signal, The electronic device according to claim 1, which performs a process of removing noise from the extracted signal components using singular value decomposition and a process of using the frequency of the extracted signal components.

3. The extraction unit, A process of extracting a micro-Doppler component of a point group where the target exists by applying a plurality of window functions to the conversion signal, A process of performing at least one of principal component analysis and singular value decomposition on the extracted micro-Doppler component, Performing frequency filtering using at least one of short-time Fourier transform, continuous wavelet transform, and band-pass filter on the result of performing at least one of the principal component analysis and singular value decomposition, The electronic device according to claim 1, wherein multi-resolution analysis including discrete wavelet transform is performed on the frequency-filtered result using a wavelet function and a scaling function appropriate for heart sounds.

4. The extraction unit determines the optimality as a heart sound signal for a signal component corresponding to the vibration accompanying the heartbeat of the target by a learning classifier, The electronic device according to claim 1, wherein an envelope signal is extracted for the signal component determined to be optimal by the learning classifier using a moving variance process.

5. The extraction unit, For the signal component, an envelope signal is extracted using any one of continuous wavelet transform, discrete wavelet transform, wavelet scattering coefficient, mel-frequency cepstral coefficient, or moving variance process, The electronic device according to claim 1, wherein for the envelope signal, an optimal envelope signal is determined based on a score calculated for the heart sound of the signal component using a trained learning classifier.

6. The extraction unit, An interval value of peak time is obtained for the extracted optimal envelope signal, and the variance and central value of the value set of the interval value are estimated, Based on the estimated variance and central value, a criterion for extracting the heart rate interval from the envelope signal is created, The electronic device according to claim 5, wherein the heart rate interval is extracted from the envelope signal using the criterion.

7. The extraction unit, Based on the extracted optimal envelope signal, the first heart sound and the second heart sound are detected, For the intervals between the first heart sound and the second heart sound, and between the second heart sound and the first heart sound, the heartbeat interval of the target is calculated by using DP matching based on the histogram or probability density distribution of the intervals, for the electronic device according to claim 6.

8. A step of transmitting a transmission wave from a transmission unit; A step of receiving a reflected wave from the target of the transmission wave upon reception; A step of detecting the distance position, direction, and speed of the target based on a conversion signal after Fourier-transforming the beat signal of the transmission wave and the reflected wave; A step of extracting heartbeat information of the target based on the detected distance, direction, and speed of the target A control method for an electronic device including: In the step of extracting the heartbeat information of the target, Signal components corresponding to vibrations associated with the heartbeat of the target are extracted from the conversion signal by a plurality of window functions centered on a plurality of distance ranges, By applying a learning classifier to the extracted signal components, the most appropriate heart sound signal is selected as the heart sound, By performing envelope processing on the most appropriate heart sound signal, the most appropriate optimal signal is selected as the heartbeat, A value indicating the interval between peak times is obtained from the optimal signal, and the heartbeat interval of the target is extracted based on the variance and central value of the value indicating the interval, for the control method of an electronic device.

9. In an electronic device, A step of transmitting a transmission wave from a transmission unit; A step of receiving a reflected wave from the target of the transmission wave upon reception; A step of detecting the distance position, direction, and speed of the target based on a conversion signal after Fourier-transforming the beat signal of the transmission wave and the reflected wave; A step of extracting heartbeat information of the target based on the detected distance, direction, and speed of the target A program for causing the above to be executed In the step of extracting the heartbeat information of the target, From the conversion signal, signal components corresponding to the vibrations associated with the heartbeat of the target are extracted by a plurality of window functions centered on a plurality of distance ranges, By applying a learning classifier to the extracted signal components, the most appropriate heart sound signal as a heart sound is selected, By performing envelope processing on the most appropriate heart sound signal, the most appropriate optimal signal as a heartbeat is selected, A program that obtains a value indicating the interval of peak times from the optimal signal and extracts the heartbeat interval of the target based on the variance and median of the values indicating the interval.

Images