Electronic apparatus, control method for electronic apparatus, and program
The electronic device uses SSNMF to process millimeter wave reflections for accurate heartbeat detection, overcoming noise interference and enabling precise monitoring in various settings.
Patent Information
- Application Number
- JP2023072698
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-26
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-04-26
AI Technical Summary
Existing technologies struggle to accurately detect heartbeats using radio waves, limiting their application in various fields that require precise heartbeat monitoring.
An electronic device employing semi-supervised non-negative matrix factorization (SSNMF) to process time-frequency analysis information from reflected millimeter waves, generating cardiac waveforms by factorizing transmission and reception signals using frequency and activation matrices.
Enables accurate detection of heartbeats with reduced noise interference, allowing for precise heartbeat monitoring in diverse environments, including moving vehicles.
Smart Images

Figure 2024157978000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an electronic device, a control method for an electronic device, and a program. [Background technology]
[0002] For example, in the field of the automobile industry, a technology for measuring the distance between a vehicle and a predetermined object is considered important. In particular, in recent years, various RADAR (Radio Detecting and Ranging) technologies have been researched, which transmit radio waves such as millimeter waves and receive the reflected waves from an object such as an obstacle to measure the distance between the vehicle and the object. The importance of such technology for measuring distance is expected to increase in the future with the development of technologies for assisting the driver in driving and technologies related to autonomous driving that automates part or all of driving.
[0003] Also, various proposals have been made regarding technologies for acquiring and analyzing sounds (bio-sounds) emitted from a living body such as the human body. For example, Patent Document 1 proposes a technology for separating and analyzing multiple types of sounds contained in bio-sounds such as breathing sounds. Patent Document 2 proposes a technology that can improve the accuracy of classifying time-series data, in which signals obtained by sensing are continuously generated, into multiple classes. Patent Document 3 proposes a signal analysis method that can learn a basis spectrum using an algorithm with guaranteed convergence. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] International Publication No. 2015 / 145754 [Patent Document 2] JP 2017-151872 A [Patent Document 3] JP 2018-136368 A Summary of the Invention [Problem to be solved by the invention]
[0005] If it were possible to detect weak vibrations such as a heartbeat with a high degree of accuracy by transmitting and receiving radio waves, such as millimeter waves, it is expected that this technology would be useful in a wide variety of fields.
[0006] An object of the present disclosure is to provide an electronic device, a control method for an electronic device, and a program that can detect a heart rate with high accuracy by transmitting and receiving radio waves. [Means for solving the problem]
[0007] The electronic device according to an embodiment includes: The monitoring device includes a signal processing unit that detects the monitoring target based on a transmission signal that is transmitted as a transmission wave and a reception signal that is received as a reflected wave of the transmission wave reflected by the monitoring target. The signal processing unit includes: The time-frequency analysis information calculated based on the transmitted wave and the reflected wave is subjected to non-negative matrix factorization using basis information of a cardiac acoustocardiogram obtained in advance. A cardiac waveform of the monitored subject is generated.
[0008] The electronic device according to an embodiment includes: The monitoring device includes a signal processing unit that detects the monitoring target based on a transmission signal that is transmitted as a transmission wave and a reception signal that is received as a reflected wave of the transmission wave reflected by the monitoring target. The signal processing unit includes: generating first time-frequency analysis information Y1 based on the transmitted wave and the reflected wave; Semi-supervised non-negative matrix factorization The first time-frequency analysis information Y1 is calculated as Y1=FG+HU using frequency basis matrices F and H and activation matrices G and U, and the frequency basis matrix F is set as a frequency basis matrix obtained from a reference cardiac sound waveform obtained in advance, and the activation matrix G is calculated using a predetermined cost function; A heart sound waveform of the monitored object is generated based on a product FG of the frequency basis matrix F and the activation matrix G.
[0009] A method for controlling an electronic device according to an embodiment includes: detecting the monitoring target based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the monitoring target; generating first time-frequency analysis information Y1 based on the transmitted wave and the reflected wave; Semi-supervised non-negative matrix factorization A step of calculating the activation matrix G using a predetermined cost function by using the first time-frequency analysis information Y1 as Y1=FG+HU using frequency basis matrices F and H and activation matrices G and U, and setting the frequency basis matrix F as a frequency basis matrix obtained from a reference heart sound waveform obtained in advance; generating a heart sound waveform of the monitored object based on a product FG of the frequency basis matrix F and the activation matrix G; Includes.
[0010] A program according to an embodiment includes: For electronic devices, detecting the monitoring target based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the monitoring target; generating first time-frequency analysis information Y1 based on the transmitted wave and the reflected wave; Semi-supervised non-negative matrix factorization A step of calculating the activation matrix G using a predetermined cost function by using the first time-frequency analysis information Y1 as Y1=FG+HU using frequency basis matrices F and H and activation matrices G and U, and setting the frequency basis matrix F as a frequency basis matrix obtained from a reference heart sound waveform obtained in advance; generating a heart sound waveform of the monitored object based on a product FG of the frequency basis matrix F and the activation matrix G; Execute the command. Effect of the Invention
[0011] According to one embodiment, it is possible to provide an electronic device, a control method for an electronic device, and a program that can detect a heart rate with high accuracy by transmitting and receiving radio waves. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a diagram illustrating a usage mode of an electronic device according to an embodiment. [Diagram 2] 1 is a functional block diagram illustrating a schematic configuration of an electronic device according to an embodiment. [Diagram 3] 1 is a diagram illustrating a configuration of a signal processed by an electronic device according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Diagram 5] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 6] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 7] 1A and 1B are diagrams illustrating an example of an antenna arrangement and an operation principle in an antenna array of an electronic device according to an embodiment. [Figure 8] 1 is a diagram illustrating an example of an arrangement of antennas in an antenna array of an electronic device according to an embodiment. [Figure 9] FIG. 2 is a diagram illustrating an example of signal processing by an electronic device according to an embodiment. [Figure 10] FIG. 2 is a diagram illustrating an example of signal processing by an electronic device according to an embodiment. [Figure 11] 10 is a flowchart illustrating a comparative example of the operation of the electronic device according to an embodiment. [Figure 12] 10 is a flowchart illustrating an operation of an electronic device according to an embodiment. [Figure 13] 10 is a flowchart illustrating an operation of an electronic device according to an embodiment. [Figure 14]FIG. 13 is a diagram illustrating, in pseudocode, the processing of semi-supervised non-negative matrix factorization by an electronic device according to an embodiment. [Figure 15] 11A to 11C are diagrams illustrating processing by an electronic device according to an embodiment using a heart sound waveform and a spectrogram. [Figure 16] FIG. 11 is a diagram illustrating an example of anomaly detection using singular spectrum transformation by an electronic device according to an embodiment. [Figure 17] FIG. 13 is a diagram illustrating, in pseudo code, a process of singular spectrum transformation by an electronic device according to an embodiment. [Figure 18] 10 is a flowchart illustrating an operation of an electronic device according to an embodiment. [Figure 19] 11A and 11B are diagrams illustrating an example of a result of signal processing by an electronic device according to an embodiment. [Figure 20] FIG. 2 is a diagram illustrating signal processing by an electronic device according to an embodiment. [Figure 21] 10 is a flowchart illustrating an operation of an electronic device according to an embodiment. [Figure 22] 11A and 11B are diagrams illustrating an example of a result of signal processing by an electronic device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, one embodiment will be described with reference to the drawings.
[0014] In the present disclosure, an "electronic device" may be a device that is powered by electricity. Furthermore, a "user" may be a person (typically a human) or an animal that uses an electronic device according to an embodiment and / or a system including the electronic device. A user may include a person who monitors a target such as a human by using an electronic device according to an embodiment. Furthermore, a "target" may be a person (a target of monitoring, for example, a human or an animal) that is monitored by an electronic device according to an embodiment. Furthermore, a user may include a target.
[0015] In this disclosure, a "heart beat" may refer to the beating of the heart. Also, a "beat" may refer to the rhythmic contraction motion of the heart. Here, the heart includes human and animal hearts. Also, a "heart beat" and a "heart sound" may include human and animal "heart beat" and "heart sound."
[0016] In addition, in this disclosure, "heart rate" refers to the number of times the heart beats within a given time. For example, the heart rate may be the number of beats per minute. When the heart pumps blood, pulsations occur in the arteries. For this reason, the number of times the arteries beat may be referred to as the "pulse rate" or simply "pulse."
[0017] Furthermore, in the present disclosure, a "heart sound" may be the sound of the heart beating. That is, a heart sound may be the sound that occurs when the heart contracts and expands. Here, the "heart sound" may be composed of a first low and long sound due to the tension of the ventricular muscles, the closing of the mitral valve, the start of blood ejection into the arteries, and / or the acceleration of blood flow, followed by a second high and short sound due to the closing of the aortic valve and / or the closing of the pulmonary valve.
[0018] In the present disclosure, heart sounds are not necessarily limited to physical sounds based on air vibrations, but may refer to the vibrations caused by the heartbeat (beating). For example, in the present disclosure, heartbeats may imply a vibration source, and heart sounds may imply the vibrations caused by the vibration source. Furthermore, in the present disclosure, heartbeats may imply heart sounds depending on the situation.
[0019] The electronic device according to an embodiment can detect the heartbeat of a target such as a human being present around the electronic device. Therefore, the electronic device according to an embodiment can be used in specific facilities used by people who engage in social activities, such as companies, hospitals, nursing homes, schools, sports gyms, and nursing homes. For example, in a company, it is very important to understand and / or manage the health of employees. Similarly, in a hospital, it is very important to understand and / or manage the health of patients and medical professionals, and in a nursing home, it is very important to understand and / or manage the health of residents and staff. The electronic device according to an embodiment can be used in any facility where it is desired to understand and / or manage the health of a target, and is not limited to the above-mentioned facilities such as companies, hospitals, and nursing homes. The facility may include, for example, a non-commercial facility such as a user's home. The electronic device according to an embodiment can be used in any facility, not limited to indoors, but may be used outdoors. For example, the electronic device according to an embodiment can be used in a moving body such as a train, bus, or airplane, as well as in a station or a boarding area. Moreover, the electronic device according to the embodiment may be used in a moving body such as an automobile, an airplane, or a ship, a hotel, a user's home, a living room in the home, a bathroom, a toilet, or a bedroom, etc. The electronic device according to the embodiment may be used to measure the heart rates of cows, pigs, or other livestock or animals in a zoo, a ranch, a farm, or the like.
[0020] The electronic device according to an embodiment may be used, for example, in a nursing facility or the like, for detecting or monitoring the heartbeat of a target such as a person requiring nursing care or a person requiring care. In addition, the electronic device according to an embodiment may issue a predetermined warning to the target and / or other people when an abnormality is found in the heartbeat of the target such as a person requiring nursing care or a person requiring care. Therefore, according to the electronic device according to an embodiment, for example, the target and / or staff of a nursing facility or the like may recognize that an abnormality is found in the pulse of a target such as a person requiring nursing care or a person requiring care. On the other hand, the electronic device according to an embodiment may notify the target and / or other people when an abnormality is not found (for example, recognized as normal) in the heartbeat of a target such as a person requiring nursing care or a person requiring care. Therefore, according to the electronic device according to an embodiment, for example, the target and / or staff of a nursing facility or the like may recognize that the pulse of a target such as a person requiring nursing care or a person requiring care is normal. An electronic device according to one embodiment may, from the heart rate of a monitored person or animal, determine the heart rate interval (RR interval = RRI in an electrocardiogram), detect dozing or other distracted driving and estimate signs of such driving, estimate the level of alertness, estimate the level of fatigue, or identify the monitored person or animal.
[0021] The electronic device according to an embodiment may detect the pulse of animals other than humans. The electronic device according to an embodiment described below will be described as detecting the pulse of a human using a sensor based on a technology such as millimeter wave radar, for example.
[0022] The electronic device according to an embodiment may be installed on any stationary object or any moving object. The electronic device according to an embodiment can transmit a transmission wave to the surroundings of the electronic device from a transmission antenna. The electronic device according to an embodiment can receive a reflected wave of the transmission wave from a reception antenna. At least one of the transmission antenna and the reception antenna may be provided in the electronic device, or may be provided in, for example, a radar sensor or the like.
[0023] Hereinafter, the electronic device according to the embodiment will be described as being stationary. The electronic device according to the embodiment may be installed in a moving body such as an automobile. The electronic device according to the embodiment may be installed in a moving body such as an automobile to detect the heartbeat of a passenger in the moving body. On the other hand, the subject (human) whose pulse is detected by the electronic device according to the embodiment may be stationary, may be moving, or may move his / her body while stationary. The electronic device according to the embodiment can measure the distance between the electronic device and an object in a situation where the object around the electronic device may move, similar to a normal radar sensor. The electronic device according to the embodiment can measure the distance between the electronic device and an object even when both the electronic device and the object are stationary.
[0024] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An electronic device according to an embodiment will be described in detail below with reference to the drawings. First, an example of object detection by the electronic device according to an embodiment will be described.
[0025] Fig. 1 is a diagram for explaining an example of a usage mode of an electronic device according to an embodiment. Fig. 1 shows an example of an electronic device having a function of a sensor and including a transmitting antenna and a receiving antenna according to an embodiment.
[0026] As shown in FIG. 1, the electronic device 1 according to an embodiment may include a transmitting unit and a receiving unit, which will be described later. As described later, the transmitting unit may include a transmitting antenna array 24. Also, the receiving unit may include a receiving antenna array 31. Specific configurations of the electronic device 1, the transmitting unit, and the receiving unit will be described later. For ease of viewing, FIG. 1 shows a state in which the electronic device 1 includes a transmitting antenna array 24 and a receiving antenna array 31. Also, the electronic device 1 may include at least one of the other functional units, such as at least a part of the signal processing unit 10 (FIG. 2) included in the electronic device 1, as appropriate. Also, the electronic device 1 may include at least one of the other functional units, such as at least a part of the signal processing unit 10 (FIG. 2) included in the electronic device 1, outside the electronic device 1. In FIG. 1, the electronic device 1 may be moving, or may be stationary without moving.
[0027] In the example shown in FIG. 1, the electronic device 1 is shown in a simplified form with a transmitter having a transmitting antenna array 24 and a receiver having a receiving antenna array 31. The electronic device 1 may include, for example, a plurality of transmitters and / or a plurality of receivers. The transmitter may include a transmitting antenna array 24 consisting of a plurality of transmitting antennas. Furthermore, the receiver may include a receiving antenna array 31 consisting of a plurality of receiving antennas. Here, the positions at which the transmitter and / or receiver are installed in the electronic device 1 are not limited to the positions shown in FIG. 1, and may be other positions as appropriate. Furthermore, the number of transmitters and / or receivers may be any number equal to or greater than one, depending on various conditions (or requirements) such as the range and / or accuracy of heartbeat detection by the electronic device 1.
[0028] As described below, the electronic device 1 transmits electromagnetic waves as transmission waves from the transmitting antenna array 24. For example, if a specific object (e.g., the target 200 shown in FIG. 1) is present around the electronic device 1, at least a part of the transmission wave transmitted from the electronic device 1 is reflected by the object and becomes a reflected wave. Then, by receiving such a reflected wave, for example, by the receiving antenna array 31 of the electronic device 1, the electronic device 1 can detect the object as a target.
[0029] The electronic device 1 equipped with the transmitting antenna array 24 may typically be a RADAR (Radio Detecting and Ranging) sensor that transmits and receives radio waves. However, the electronic device 1 is not limited to a radar sensor. The electronic device 1 according to an embodiment may be a sensor based on, for example, a LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) technology using light waves. Such sensors may be configured to include, for example, a patch antenna. Since technologies such as RADAR and LIDAR are already known, detailed descriptions thereof may be appropriately simplified or omitted. Furthermore, the electronic device 1 according to an embodiment may be a sensor based on a technology that detects an object by transmitting and receiving, for example, sound waves or ultrasonic waves.
[0030] The electronic device 1 shown in FIG. 1 receives, from the receiving antenna array 31, a reflected wave of a transmission wave transmitted from the transmitting antenna array 24. In this way, the electronic device 1 can detect a predetermined object 200 existing within a predetermined distance from the electronic device 1 as a target. For example, as shown in FIG. 1, the electronic device 1 can measure a distance L between the electronic device 1 and the predetermined object 200. The electronic device 1 can also measure the relative speed between the electronic device 1 and the predetermined object 200. Furthermore, the electronic device 1 can also measure the direction (arrival angle θ) in which the reflected wave from the predetermined object 200 arrives at the electronic device 1.
[0031] In Fig. 1, the XY plane may be, for example, a plane that is approximately parallel to the ground surface. In this case, the positive direction of the Z axis shown in Fig. 1 may indicate a vertically upward direction. In Fig. 1, the electronic device 1 may be disposed on a plane that is parallel to the XY plane. Also, in Fig. 1, the target 200 may be, for example, standing on the ground surface that is approximately parallel to the XY plane.
[0032] Here, the target 200 may be, for example, a human being present around the electronic device 1. The target 200 may also be a living thing other than a human being, such as an animal present around the electronic device 1. As described above, the target 200 may be moving, stopped, or stationary. In the present disclosure, the object detected by the electronic device 1 includes inanimate objects such as any object, as well as living things such as people, dogs, cats, horses, and other animals. The object detected by the electronic device 1 of the present disclosure may include targets including people, objects, and animals detected by radar technology. In the present disclosure, targets may include people, objects, and animals. In the following, the description will be given on the assumption that an object such as the target 200 present around the electronic device 1 is a human being (or an animal). Hereinafter, the "target 200" may also be appropriately written as the "subject 200". In the present disclosure, the target may be the above-mentioned target 200.
[0033] 1, the ratio between the size of the electronic device 1 and the size of the target 200 does not necessarily indicate the actual ratio. Also, in FIG. 1, the transmitting antenna array 24 of the transmitting unit and the receiving antenna array 31 of the receiving unit are shown installed outside the electronic device 1. However, in one embodiment, the transmitting antenna array 24 of the transmitting unit and / or the receiving antenna array 31 of the receiving unit may be installed at various positions of the electronic device 1. For example, in one embodiment, the transmitting antenna array 24 of the transmitting unit and / or the receiving antenna array 31 of the receiving unit may be installed inside the electronic device 1 so as not to be visible from the outside of the electronic device 1.
[0034] In the following, as a typical example, the transmitting antenna of the electronic device 1 will be described as transmitting radio waves in a frequency band such as millimeter waves (30 GHz or higher) or quasi-millimeter waves (for example, around 20 GHz to 30 GHz). On the other hand, the transmitting antenna of the electronic device 1 may transmit radio waves having a frequency bandwidth of 4 GHz, such as 77 GHz to 81 GHz. In addition, the transmitting antenna of the electronic device 1 may transmit radio waves of a high frequency (for example, 30 GHz to 300 GHz) equal to or higher than the millimeter wave band.
[0035] 2 is a functional block diagram illustrating an example of the configuration of the electronic device 1 according to an embodiment. An example of the configuration of the electronic device 1 according to an embodiment will be described below.
[0036] When measuring distances and the like using a millimeter wave radar, a frequency modulated continuous wave radar (hereinafter, referred to as FMCW radar) is often used. In an FMCW radar, a transmission signal is generated by sweeping the frequency of radio waves to be transmitted. Therefore, in a millimeter wave FMCW radar using radio waves in a frequency band of, for example, 79 GHz, the frequency of the radio waves used has a frequency bandwidth of 4 GHz, for example, 77 GHz to 81 GHz. A radar in the 79 GHz frequency band has a feature that the available frequency bandwidth is wider than other millimeter wave / quasi-millimeter wave radars in frequency bands of, for example, 24 GHz, 60 GHz, and 76 GHz. Hereinafter, such an embodiment will be described as an example.
[0037] The FMCW radar system used in the present disclosure may include an FCM (Fast-Chirp Modulation) system that transmits a chirp signal at a shorter period than normal. The signal generated by the electronic device 1 is not limited to an FMCW signal. The signal generated by the electronic device 1 may be a signal of various systems other than the FMCW system. The transmission signal sequence stored in an arbitrary storage unit may differ depending on these various systems. For example, in the case of the radar signal of the above-mentioned FMCW system, a signal whose frequency increases and decreases for each time sample may be used. Since the above-mentioned various systems can be appropriately applied using known technologies, a more detailed description will be omitted.
[0038] As shown in FIG. 2, the electronic device 1 according to the embodiment includes a signal processing unit 10. The signal processing unit 10 may include a signal generation processing unit 11, a reception signal processing unit 12, a heartbeat extraction unit 13, and a calculation unit 14. The heartbeat extraction unit 13 may execute, for example, a process of extracting a micro-Doppler component. The heartbeat extraction unit 13 may also execute a process of extracting an envelope of the heart sounds of the subject 200. The calculation unit 14 may execute, for example, a process of calculating the heartbeat interval (RRI) of the subject 200. The calculation unit 14 may also execute, for example, a process of calculating the heartbeat of the subject 200. The calculation unit 14 may also execute, for example, a process of calculating the heart rate validity (HRV) of the subject 200. In this case, the calculation unit 14 may execute a process of performing a frequency analysis of the time series data of the heartbeat interval of the subject 200 that has been extracted. The calculation unit 14 may also execute a process of calculating the heartbeat variability of the subject 200 based on the frequency analysis of the time series data of the heartbeat interval. The signal generation processing unit 11, the received signal processing unit 12, the heartbeat extraction unit 13, and the calculation unit 14 will be described further below as appropriate. In the present disclosure, the heart sound may be, for example, a chest vibration waveform directly observed by a radar, or may be chest vibration. The heartbeat is the heartbeat itself. The heartbeat interval, the heart rate, etc. may be calculated from the movement of the heartbeat. The heartbeat interval may be the time interval between one heartbeat and the next heartbeat.
[0039] The electronic device 1 according to an embodiment includes a transmission DAC 21, a transmission circuit 22, a millimeter wave transmission circuit 23, and a transmission antenna array 24 as a transmission unit. The electronic device 1 according to an embodiment includes a reception antenna array 31, a mixer 32, a reception circuit 33, and a reception ADC 34 as a reception unit. The electronic device 1 according to an embodiment may not include at least one of the functional units shown in FIG. 2, or may include a functional unit other than the functional unit shown in FIG. 2. The electronic device 1 shown in FIG. 2 may be configured using a circuit basically configured similarly to a general radar using electromagnetic waves such as a millimeter wave band. On the other hand, in the electronic device 1 according to an embodiment, the signal processing by the signal processing unit 10 may include processing different from that of a conventional general radar.
[0040] The signal processing unit 10 included in the electronic device 1 according to an embodiment can control the operation of the entire electronic device 1, including the control of each functional unit constituting the electronic device 1. In particular, the signal processing unit 10 performs various processes on signals handled by the electronic device 1. The signal processing unit 10 may include at least one processor, such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), to provide control and processing power for executing various functions. The signal processing unit 10 may be realized as one processor, several processors, or individual processors. The processor may be realized as a single integrated circuit. The integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as a plurality of integrated circuits and discrete circuits connected to each other so as to be able to communicate with each other. The processor may be realized based on various other known technologies. In an embodiment, the signal processing unit 10 may be configured as, for example, a CPU (hardware) and a program (software) executed by the CPU. The signal processing unit 10 may include a storage unit (memory) necessary for the operation of the signal processing unit 10 as appropriate.
[0041] The signal generation processing unit 11 of the signal processing unit 10 generates a signal to be transmitted from the electronic device 1. In the electronic device 1 according to an embodiment, the signal generation processing unit 11 may generate a transmission signal (transmission chirp signal) such as a chirp signal. In particular, the signal generation processing unit 11 may generate a signal (linear chirp signal) whose frequency changes periodically and linearly. For example, the signal generation processing unit 11 may generate a chirp signal whose frequency increases periodically and linearly from 77 GHz to 81 GHz over time. Also, for example, the signal generation processing unit 11 may generate a signal whose frequency periodically repeats a linear increase (up chirp) and decrease (down chirp) from 77 GHz to 81 GHz over time. The signal generated by the signal generation processing unit 11 may be set in advance in, for example, the signal processing unit 10. Also, the signal generated by the signal generation processing unit 11 may be stored in advance in, for example, an arbitrary storage unit in the signal processing unit 10. Since chirp signals used in technical fields such as radar are known, a more detailed description will be appropriately simplified or omitted. The signal generated by the signal generating and processing unit 11 is supplied to the transmitting DAC 21. For this reason, the signal generating and processing unit 11 may be connected to the transmitting DAC 21.
[0042] The transmission DAC (digital-analog converter) 21 has a function of converting the digital signal supplied from the signal generating processing unit 11 into an analog signal. The transmission DAC 21 may be configured to include a general digital-analog converter. The signal converted into an analog signal by the transmission DAC 21 is supplied to the transmission circuit 22. For this reason, the transmission DAC 21 may be connected to the transmission circuit 22.
[0043] The transmission circuit 22 has a function of converting the signal converted into an analog signal by the transmission DAC 21 into an intermediate frequency (IF) band. The transmission circuit 22 may be configured to include a general IF band transmission circuit. The signal processed by the transmission circuit 22 is supplied to the millimeter wave transmission circuit 23. For this reason, the transmission circuit 22 may be connected to the millimeter wave transmission circuit 23.
[0044] The millimeter wave transmission circuit 23 has a function of transmitting the signal processed by the transmission circuit 22 as a millimeter wave (RF wave). The millimeter wave transmission circuit 23 may be configured to include a general millimeter wave transmission circuit. The signal processed by the millimeter wave transmission circuit 23 is supplied to the transmission antenna array 24. Therefore, the millimeter wave transmission circuit 23 may be connected to the transmission antenna array 24. The signal processed by the millimeter wave transmission circuit 23 is also supplied to the mixer 32. Therefore, the millimeter wave transmission circuit 23 may also be connected to the mixer 32.
[0045] The transmitting antenna array 24 is an array of a plurality of transmitting antennas. In Fig. 2, the configuration of the transmitting antenna array 24 is shown in a simplified form. The transmitting antenna array 24 transmits the signal processed by the millimeter wave transmitting circuit 23 to the outside of the electronic device 1. The transmitting antenna array 24 may be configured to include a transmitting antenna array used in a general millimeter wave radar.
[0046] In this manner, the electronic device 1 according to the embodiment includes a transmitting antenna (the transmitting antenna array 24), and can transmit a transmitting signal (for example, a transmitting chirp signal) from the transmitting antenna array 24 as a transmitting wave.
[0047] 2, it is assumed that an object such as a subject 200 is present around the electronic device 1. In this case, at least a part of the transmission wave transmitted from the transmitting antenna array 24 is reflected by the object such as the subject 200. At least a part of the transmission wave transmitted from the transmitting antenna array 24 that is reflected by the object such as the subject 200 may be reflected toward the receiving antenna array 31.
[0048] The receiving antenna array 31 receives the reflected waves, which may be at least a portion of the transmitted waves transmitted from the transmitting antenna array 24 that are reflected by an object such as the subject 200.
[0049] The receiving antenna array 31 is an array of multiple receiving antennas. In Fig. 2, the configuration of the receiving antenna array 31 is shown in a simplified form. The receiving antenna array 31 receives reflected waves of the transmission waves transmitted from the transmitting antenna array 24. The receiving antenna array 31 may be configured to include a receiving antenna array used in a general millimeter wave radar. The receiving antenna array 31 supplies a reception signal received as a reflected wave to the mixer 32. For this reason, the receiving antenna array 31 may be connected to the mixer 32.
[0050] The mixer 32 converts the signal (transmission signal) processed by the millimeter wave transmission circuit 23 and the reception signal received by the reception antenna array 31 into an intermediate frequency (IF) band. The mixer 32 may be configured to include a mixer used in a typical millimeter wave radar. The mixer 32 supplies a signal generated as a result of the synthesis to the reception circuit 33. For this reason, the mixer 32 may be connected to the reception circuit 33.
[0051] The receiving circuit 33 has a function of performing analog processing on the signal converted to the IF band by the mixer 32. The receiving circuit 33 may be configured to include a receiving circuit that converts to a general IF band. The signal processed by the receiving circuit 33 is supplied to the receiving ADC 34. For this reason, the receiving circuit 33 may be connected to the receiving ADC 34.
[0052] The receiving ADC (analog-digital converter) 34 has a function of converting an analog signal supplied from the receiving circuit 33 into a digital signal. The receiving ADC 34 may be configured to include a general analog-digital converter. The signal digitized by the receiving ADC 34 is supplied to the receiving signal processing unit 12 of the signal processing unit 10. For this reason, the receiving ADC 34 may be connected to the signal processing unit 10.
[0053] The reception signal processing unit 12 of the signal processing unit 10 has a function of performing various processes on the digital signal supplied from the reception ADC 34. For example, the reception signal processing unit 12 calculates the distance from the electronic device 1 to an object such as the subject 200 based on the digital signal supplied from the reception ADC 34 (distance measurement). The reception signal processing unit 12 also calculates the relative speed of the object such as the subject 200 with respect to the electronic device 1 based on the digital signal supplied from the reception ADC 34 (speed measurement). Furthermore, the reception signal processing unit 12 calculates the azimuth angle of the object such as the subject 200 as seen from the electronic device 1 based on the digital signal supplied from the reception ADC 34 (angle measurement). Specifically, I / Q converted data may be input to the reception signal processing unit 12. By inputting such data, the reception signal processing unit 12 performs fast Fourier transform (2D-FFT) in the range direction and the velocity direction, respectively. Thereafter, the reception signal processing unit 12 may suppress false alarms by removing noise points through processing such as CFAR (Constant False Alarm Rate) and make the probability constant. Then, the reception signal processing unit 12 can obtain the position of an object such as the subject 200 by estimating the arrival angle for points that satisfy the CFAR criteria. Information generated as a result of distance, speed, and angle measurement by the reception signal processing unit 12 may be supplied to the heart rate extraction unit 13.
[0054] The heartbeat extraction unit 13 extracts information related to the heartbeat from the information generated by the reception signal processing unit 12. The operation of extracting information related to the heartbeat by the heartbeat extraction unit 13 will be described further below. The information related to the heartbeat extracted by the heartbeat extraction unit 13 may be supplied to the calculation unit 14.
[0055] The calculation unit 14 performs various calculation processes and / or arithmetic processes on the information on the heartbeat supplied from the heartbeat extraction unit 13. The various calculation processes and / or arithmetic processes performed by the calculation unit 14 will be described further below. The various information calculated and / or processed by the calculation unit 14 may be supplied to, for example, a communication interface 50. For this reason, the calculation unit 14 and / or the signal processing unit 10 may be connected to the communication interface 50. The various information calculated and / or processed by the calculation unit 14 may be supplied to other functional units other than the communication interface 50.
[0056] The communication interface 50 includes an interface that outputs information supplied from the signal processing unit 10 to, for example, an external device 60. The communication interface 50 may output at least one of information regarding the position, speed, and angle of an object such as the subject 200 to the external device 60 as, for example, a signal such as a CAN (Controller Area Network). For example, at least one of information regarding the position, speed, and angle of an object such as the subject 200 may be supplied to the external device 60 via the communication interface 50. For this reason, the communication interface 50 may be connected to the external device 60.
[0057] As shown in Fig. 2, the electronic device 1 according to an embodiment may be connected to an external device 60 via a communication interface 50 in a wired or wireless manner. In an embodiment, the external device 60 may be configured to include any computer and / or any control device. The electronic device 1 according to an embodiment may also be configured to include the external device 60. The external device 60 may have various configurations depending on how the information on the heartbeat and / or heart sounds detected by the electronic device 1 is used. Therefore, a more detailed description of the external device 60 will be omitted.
[0058] FIG. 3 is a diagram for explaining an example of a chirp signal generated by the signal generation processing unit 11 of the signal processing unit 10. In FIG.
[0059] FIG. 3 shows the time structure of one frame when the FCM (Fast-Chirp Modulation) method is used. FIG. 3 shows an example of a received signal in the FCM method. FCM is a method in which chirp signals shown as c1, c2, c3, c4, ..., cn in FIG. 3 are repeated at relatively short intervals (for example, longer than the round-trip time between the electromagnetic radar and the target calculated from the maximum ranging distance). In FCM, for convenience of signal processing of the received signal, transmission and reception processing is often performed by dividing it into subframe units as shown in FIG. 3.
[0060] In Fig. 3, the horizontal axis represents the elapsed time, and the vertical axis represents the frequency. In the example shown in Fig. 3, the signal generation processing unit 11 generates a linear chirp signal whose frequency changes periodically and linearly. In Fig. 3, each chirp signal is shown as c1, c2, c3, c4, ..., cn. As shown in Fig. 3, in each chirp signal, the frequency increases linearly with the passage of time.
[0061] In the example shown in FIG. 3, several chirp signals such as c1, c2, c3, c4, ..., cn are included to form one subframe. That is, subframe 1 and subframe 2 shown in FIG. 3 are each configured to include several chirp signals such as c1, c2, c3, c4, ..., cn. Also, in the example shown in FIG. 3, several subframes such as subframe 1, subframe 2, ..., subframe N are included to form one frame (1 frame). That is, one frame shown in FIG. 3 is configured to include N subframes. Also, one frame shown in FIG. 3 may be frame 1, followed by frame 2, frame 3, ..., etc. Each of these frames may be configured to include N subframes, similar to frame 1. Also, a frame interval of a predetermined length may be included between the frames. One frame shown in FIG. 3 may be, for example, about 30 milliseconds to 50 milliseconds long.
[0062] In the electronic device 1 according to an embodiment, the signal generation processing unit 11 may generate a transmission signal as an arbitrary number of frames. Also, some chirp signals are omitted in Fig. 3. In this manner, the relationship between the time and frequency of the transmission signal generated by the signal generation processing unit 11 may be stored as various setting parameters in, for example, a storage unit of the signal processing unit 10.
[0063] In this manner, the electronic device 1 according to an embodiment may transmit a transmission signal consisting of subframes including a plurality of chirp signals. Also, the electronic device 1 according to an embodiment may transmit a transmission signal consisting of a frame including a predetermined number of subframes.
[0064] Hereinafter, the electronic device 1 will be described as transmitting a transmission signal having a frame structure as shown in FIG. 3. However, the frame structure as shown in FIG. 3 is an example, and for example, the chirp signal included in one subframe may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate a subframe including an arbitrary number (for example, an arbitrary multiple) of chirp signals. Also, the subframe structure as shown in FIG. 3 is an example, and for example, the subframe included in one frame may be arbitrary. That is, in one embodiment, the signal generation processing unit 11 may generate a frame including an arbitrary number (for example, an arbitrary multiple) of subframes. The signal generation processing unit 11 may generate signals of different frequencies. The signal generation processing unit 11 may generate a plurality of discrete signals having different bandwidths of frequency f.
[0065] Fig. 4 is a diagram showing, in another aspect, a part of the subframe shown in Fig. 3. Fig. 4 shows each sample of the received signal that has received the transmission signal shown in Fig. 3 as a result of 2D-FFT (Two Dimensional Fast Fourier Transform) processing performed by the received signal processing unit 12 (Fig. 2) of the signal processing unit 10.
[0066] As shown in Fig. 4, chirp signals c1, c2, c3, c4, ..., cn are stored in each subframe such as subframe 1, ..., subframe N. In Fig. 4, each chirp signal c1, c2, c3, c4, ..., cn is composed of samples represented by squares arranged in the horizontal direction. The received signal shown in Fig. 4 is subjected to 2D-FFT, CFAR, and / or integrated signal processing of each subframe by the received signal processing unit 12 shown in Fig. 2.
[0067] FIG. 5 is a diagram showing an example in which a group of points on a range-Doppler (distance-velocity) plane is calculated as a result of 2D-FFT, CFAR, and integrated signal processing of each subframe being performed in the received signal processing unit 12 shown in FIG. 2.
[0068] In FIG. 5, the horizontal direction represents range (distance) and the vertical direction represents velocity. The filled squares s1 shown in FIG. 5 represent point clouds indicating signals that exceed the CFAR threshold processing. The unfilled squares s2 shown in FIG. 5 represent bins (2D-FFT samples) without point clouds that do not exceed the CFAR threshold. The point clouds on the range-Doppler plane calculated in FIG. 5 have their orientation from the radar calculated by direction estimation, and the position and velocity on a two-dimensional plane are calculated as a point cloud indicating an object such as the subject 200. Here, the direction estimation may be calculated by a beamformer and / or a subspace method. Representative subspace method algorithms include MUSIC (MUltiple SIgnal Classification) and ESPRIT (Estimation of Signal Parameters via Rotation Invariance Technique).
[0069] Fig. 6 is a diagram showing an example of the result of conversion of point cloud coordinates from the range-Doppler plane shown in Fig. 5 to the XY plane after the received signal processing unit 12 performs direction estimation. The XY plane shown in Fig. 6 may be the same as the XY plane shown in Fig. 1. As shown in Fig. 6, the received signal processing unit 12 can plot a point cloud PG on the XY plane. Here, the point cloud PG is composed of each point P. Furthermore, each point P has an angle θ and a radial velocity Vr in polar coordinates.
[0070] The reception signal processing unit 12 detects an object present within the range where the transmission wave is transmitted based on at least one of the results of the 2D-FFT and the angle estimation. The reception signal processing unit 12 may perform object detection by, for example, clustering processing based on the estimated distance information, speed information, and angle information. As an algorithm used for clustering data, for example, DBSCAN (Density-based spatial clustering of applications with noise) is known. This is an algorithm that performs clustering based on density. In the clustering processing, for example, the average power of the points constituting the detected object may be calculated. The distance information, speed information, angle information, and power information of the object detected in the reception signal processing unit 12 may be supplied to an external device 60 or the like via a communication interface 50, for example.
[0071] As described above, the electronic device 1 may include a transmitting antenna (transmitting antenna array 24), a receiving antenna (receiving antenna array 31), and a signal processing unit 10. The transmitting antenna array 24 transmits a transmission wave, for example, an electric wave. The receiving antenna array 31 receives a reflected wave of the transmission wave. The signal processing unit 10 detects an object (for example, an object such as the subject 200) that reflects the transmission wave, based on the transmission signal transmitted as the transmission wave and the reception signal received as the reflected wave.
[0072] Next, the estimation of the direction of an incoming wave (an incoming reflected wave) by the antenna array of the electronic device 1 according to an embodiment will be further described.
[0073] 7 is a diagram for explaining the configuration of the receiving antenna array 31 of the electronic device 1 according to an embodiment, and the principle of estimating the direction of an incoming wave by the receiving antenna array 31. FIG. 7 shows an example of reception of radio waves by the receiving antenna array 31.
[0074] As shown in Fig. 7, the receiving antenna array 31 may be a linear arrangement of sensors such as receiving antennas. As shown in Fig. 7, in one embodiment, the receiving antenna array 31 may include multiple receiving antennas arranged in a linear arrangement. In Fig. 7, the receiving antenna array 31 includes antennas x1, x2, x3, ..., x M A plurality of antennas such as the above are indicated by small circles. The receiving antenna array 31 may be composed of any number of antennas. As shown in FIG. 7, the plurality of antennas constituting the receiving antenna array 31 are arranged at an interval of the array pitch d. A sensor array in which sensors (antennas, ultrasonic transducers, microphones, etc.) corresponding to various physical waves are arranged in an array is also called a Uniform Linear Array (ULA). As shown in FIG. 7, physical waves (electromagnetic waves, sound waves, etc.) arrive from various directions such as θ1 and θ2. Here, θ1 and θ2 may be the above-mentioned angles of arrival. In this way, a sensor array such as the receiving antenna array 31 can estimate the direction of arrival (angle of arrival) by utilizing the phase difference that occurs in the measured values between the sensors according to the direction of arrival of the physical wave. A method of estimating the direction of arrival of a wave in this way is also referred to as angle of arrival estimation or direction of arrival (DoA).
[0075] In the electronic device 1 according to an embodiment, at least one of the transmitting antenna array 24 and the receiving antenna array 31 may be a linear array of multiple antennas. This allows, for example, a millimeter wave radar to appropriately narrow the directivity when transmitting and receiving radio waves. When transmitting a transmission wave, the direction of the transmission beam is often controlled by a beamformer. On the other hand, when receiving a reflected wave, the arrival direction of the reflected wave is often estimated by a subspace method (such as the above-mentioned MUSIC and ESPRIT) rather than by a beamformer. In the beamformer and subspace method, in a ULA as shown in FIG. 7, a phase difference occurs in the measurement values between sensors depending on the arrival direction of electromagnetic waves arriving from various directions. Therefore, the arrival direction of the reflected wave can be estimated by utilizing the phase difference.
[0076] Next, the estimation of angles of incoming waves in two directions by the antenna array of the electronic device 1 according to an embodiment will be further described.
[0077] FIG. 8 is a diagram showing an example of an antenna arrangement for estimating directions of arrival at two orthogonal angles.
[0078] As shown in FIG. 8, in the electronic device 1 according to an embodiment, the transmitting antenna array 24 and / or the receiving antenna array 31 may include an array of a plurality of patch antenna units.
[0079] In the transmitting antenna array 24 shown in FIG. 8, one patch antenna unit may be configured to include a plurality of elements electrically connected in a direction 1 shown in the figure. In each patch antenna unit, the plurality of elements may be electrically connected by wiring such as a strip line on a substrate. In each patch antenna unit, each of the plurality of elements is spaced apart at an interval d shorter than half the wavelength λ of the transmission wave. 1,t 8, each patch antenna unit may have any number of elements, two or more, electrically connected.
[0080] 8, the transmitting antenna array 24 may be configured by arraying a plurality of patch antenna units in a direction 2 shown in the figure. The patch antenna units are spaced apart from each other by an interval d shorter than half the wavelength λ of the transmitting wave. 2,t In one embodiment, the transmit antenna array 24 may include any number of patch antenna units greater than or equal to two.
[0081] As shown in Fig. 8, in one embodiment, the receiving antenna array 31 may be a modified version of the arrangement of the elements in the transmitting antenna array 24. That is, in the receiving antenna array 31 shown in Fig. 8, one patch antenna unit may be configured to include a plurality of elements electrically connected in the direction 2 shown in the figure. In each patch antenna unit, the plurality of elements may be electrically connected by wiring such as a strip line on a substrate. In each patch antenna unit, each of the plurality of elements is spaced apart at an interval d shorter than half the wavelength λ of the transmission wave. 2,s 8, each patch antenna unit may have any number of elements, two or more, electrically connected.
[0082] 8, the receiving antenna array 31 may be configured by arraying a plurality of patch antenna units in a direction 1 shown in the figure. The patch antenna units are spaced apart from each other by an interval d shorter than half the wavelength λ of the transmission wave. 1,s In one embodiment, the receive antenna array 31 may include any number of patch antenna units greater than or equal to two.
[0083] All elements included in the transmitting antenna array 24 and the receiving antenna array 31 may be arranged on the same plane (for example, on the surface layer of the same substrate). The transmitting antenna array 24 and the receiving antenna array 31 may be arranged close to each other (monostatic). Furthermore, the direction 1 and the direction 2 shown in FIG. 8 may be geometrically orthogonal to each other.
[0084] The directivity of each of the transmitting antenna and the receiving antenna can be appropriately narrowed by the transmitting antenna array 24 and the receiving antenna array 31 as shown in FIG. 8. In addition, by using the transmitting antenna array 24 as shown in FIG. 8, the direction of transmission of each transmitting wave (transmitting signal) can be controlled at each timing of transmitting the transmitting wave, thereby realizing a beamformer for the direction 2 shown in FIG. 8. Furthermore, by using the receiving antenna array 31 as shown in FIG. 8, the arrival direction of the reflected wave can be estimated for the direction 1 shown in FIG. 8. In this way, it is possible to estimate the arrival direction of the reflected wave for two angles that are substantially orthogonal. Therefore, it is possible to obtain a point cloud showing an object such as the subject 200 in three dimensions.
[0085] Next, a method for detecting the heartbeat of the subject 200 using the electronic device 1 according to an embodiment will be described.
[0086] The electronic device 1 according to an embodiment transmits a transmission wave such as a millimeter wave radar to the subject 200, and measures (estimates) the heartbeat of the subject 200 based on the result of receiving the reflected wave reflected at the chest where the heart of the subject 200 is located. As described above, the subject 200 may be a human or an animal. In this case, for example, a component assumed to be the envelope of the heartbeat can be extracted by frequency filtering the vibration at the position where the subject 200 is present, which is detected by the radar. Then, when the component assumed to be the envelope of the heartbeat is extracted, the interval between the peaks of the envelope is taken as the heartbeat interval, and an approximate heartbeat interval can be calculated. Here, an approximation can be used in which the peak of the heartbeat envelope roughly coincides with the R peak of the electrocardiogram. For this reason, the "heartbeat interval" is also written as the RR interval or RRI (RR interval), similar to the term used for the electrocardiogram.
[0087] Here, a method for estimating the heartbeat interval of subject 200 from the results of the above-mentioned 2D-FFT, CFAR processing, and arrival direction estimation will be considered. First, the appearance of a person's heartbeat or body movement will be described as a result of the 2D-FFT performed in Figs. 4 and 5.
[0088] FIG. 9 is a diagram showing an example of a result of receiving a reflected wave of a transmission wave transmitted to the subject 200 and performing 2D-FFT processing. FIG. 9 shows a spectrum showing the heart sound and body movement of the subject 200 as a result of 2D-FFT. In FIG. 9, the horizontal axis represents distance (Range) and the vertical axis represents velocity (Velocity). The signal processing unit 10 (e.g., heartbeat extraction unit 13) of the electronic device 1 according to an embodiment may extract, for example, a peak Hm as shown in FIG. 9 as a body movement such as the heartbeat of the subject 200. Here, the spectrum components shown by the peak Hm in FIG. 9 include not only the heart sound and the envelope of the heart sound of the subject 200 but also the body movement. In order to extract the heartbeat interval, it is necessary to extract the body movement, etc., so it is assumed that, for example, frequency filtering is performed. The frequency filtering performed here is assumed to be, for example, a band pass filter, a high pass filter, and / or a low pass filter for 0.5 Hz to 10 Hz.
[0089] FIG. 10 is a diagram for explaining a method of detecting a peak based on the envelope waveform of the heart sound obtained by the above-mentioned frequency filtering. The graph shown in FIG. 10 shows an example of a time change of the envelope waveform of the heart sound extracted by the above-mentioned frequency filtering. A heart sound waveform as shown in FIG. 10 can be obtained by a method of restoring a time waveform from the result of frequency analysis such as an inverse Fourier transform performed on the 2D-FFT data shown in FIG. 9. The envelope waveform shown in FIG. 10 includes many peaks as shown. On the other hand, it is known that the heartbeat of the subject 200 falls within a range of about 50 to 130 times per minute. Therefore, by selecting a peak having a time interval of 0.4 seconds to 0.8 seconds, which is the reciprocal of the number of heartbeats per minute, from among the many peaks as shown in FIG. 10, it is possible to calculate an approximate heart sound interval of the subject 200. For example, the peak indicated by the downward arrow in FIG. 10 may be selected as the approximate heart sound interval of the subject 200. The heart rate of subject 200 can be calculated from the intervals between heartbeats, such as those seen in the section including multiple peaks indicated by downward arrows in FIG.
[0090] 11 is a flowchart showing an example of the above-mentioned operation of estimating a heartbeat interval. Hereinafter, the above-mentioned operation of estimating a heartbeat interval will be outlined with reference to FIG.
[0091] Fig. 11 shows the operation of the electronic device 1 according to an embodiment after receiving a reflected wave. That is, as a premise for the operation shown in Fig. 11, the electronic device 1 shown in Fig. 2 transmits a transmission wave (transmission signal) from the transmission antenna array 24. Then, at least a part of the transmission wave transmitted from the electronic device 1 is reflected by the subject 200 (for example, the chest) to become a reflected wave. Then, the electronic device 1 shown in Fig. 2 receives such a reflected wave from the reception antenna array 31 and converts it into a digital signal (reception ADC 34). Then, the operation shown in Fig. 11 starts.
[0092] 11 starts, first, in step S110, the signal processing unit 10 of the electronic device 1 processes a received signal (received signal). The signal processing performed in step S110 may include, for example, the above-mentioned 2D-FFT, CFAR processing, and / or arrival direction estimation. Such an operation may be performed, for example, by the received signal processing unit 12 of the signal processing unit 10.
[0093] Next, in step S120, the signal processing unit 10 extracts a signal source from the information processed in step S110. The signal processing performed in step S120 may include, for example, a process of filtering data resulting from 2D-FFT processing. In step S120, the signal processing unit 10 may extract spectral components only at the position where the subject 200 is present. Here, the position where the subject 200 is present may be specified by various known methods. Such an operation may be performed, for example, by the heart rate extraction unit 13 of the signal processing unit 10.
[0094] Next, in step S130, the signal processing unit 10 converts the result of the previous processing into a vibration waveform. The processing performed in step S130 may include, for example, a process of extracting phase information from the IQ data. In step S130, the signal processing unit 10 may include a process of extracting vibration data including heart sounds from the spectral components of the 2D-FFT processing of the subject 200 extracted in step S120. Such an operation may be performed by, for example, the heartbeat extraction unit 13 of the signal processing unit 10.
[0095] Next, in step S140, the signal processing unit 10 extracts vibration data from the result of the previous processing. The processing performed in step S140 may include, for example, frequency filtering. In step S140, the signal processing unit 10 may extract a low-frequency signal including an envelope of the heart sounds of the subject 200 by performing frequency filtering. Such an operation may be performed, for example, by the heartbeat extraction unit 13 of the signal processing unit 10.
[0096] Next, in step S150, the signal processing unit 10 detects peaks of the heartbeat of the subject 200 from the results of the previous processing, and calculates the intervals between the peaks to calculate the RR interval (RRI) of the subject 200. In step S150, the signal processing unit 10 may detect peaks of a low-frequency signal including an envelope of the heart sounds of the subject 200. Also, in step S150, the signal processing unit 10 may calculate and / or extract the intervals between the respective peaks. Such an operation may be performed, for example, by the calculation unit 14 of the signal processing unit 10. As described above, in step S150, the signal processing unit 10 may extract the intervals between the heart sounds.
[0097] Next, in step S160, the signal processing unit 10 may calculate the heart rate variability (HRV) of the subject 200. In step S160, the signal processing unit 10 may calculate the HRV of the subject 200 by calculating the spectral density of the time series data of the RRI. When calculating the power spectral density of the time series data of the RRI, a frequency analysis of the time series waveform of the RRI may be performed, for example, by using the Welch method. Such an operation may be performed, for example, by the calculation unit 14 of the signal processing unit 10. Also, in step S160, the signal processing unit 10 may analyze the spectrum from the processing result of the previous stage.
[0098] As described above, the electronic device 1 according to an embodiment extracts a component that is thought to be the envelope of the heartbeat by frequency filtering the vibration at the position where the subject 200 is present, and determines the interval between the peaks of the envelope as the heartbeat interval. In this way, the electronic device 1 according to an embodiment can calculate the approximate heartbeat interval of the subject 200.
[0099] For example, the electronic device 1 according to an embodiment can detect the heart rate of a passenger in a moving body such as an automobile by being installed in the moving body. In this case, the electronic device 1 according to an embodiment can analyze the RRI of a driver by measuring the heart rate of the driver who is operating the moving body such as an automobile using a radar. If such an RRI analysis can be performed with relatively high accuracy, the electronic device 1 can be used for applications such as estimating the psychological state of a passenger such as a driver, including the degree of alertness. In addition, if such an RRI analysis can be performed with relatively high accuracy, the electronic device 1 can be used for applications such as detecting the driver's mindless driving, including drowsy driving, or detecting a sign of such mindless driving.
[0100] On the other hand, when a moving body such as an automobile in which the electronic device 1 is installed travels on a road, various vibrations including road noise caused by unevenness of the road surface frequently occur. In such a situation, vibrations other than pulsation are superimposed as noise on the skin surface of a passenger such as a driver of the moving body such as an automobile. Therefore, while such a moving body is traveling, the accuracy of detecting the passenger's heartbeat by the electronic device 1 may be affected.
[0101] Therefore, the electronic device 1 according to an embodiment further improves on the above-mentioned method. The electronic device 1 according to an embodiment extracts heart sounds using, for example, a radar using a high frequency band of millimeter waves or higher, and analyzes the heart sounds to accurately extract heartbeat intervals. The electronic device 1 according to an embodiment also extracts a waveform with high accuracy even from a heart sound waveform on which noise, including road noise, is superimposed.
[0102] The electronic device 1 according to an embodiment executes signal processing including Fourier transform in an appropriate order based on, for example, transmitted waves and reflected waves of a millimeter wave radar. The electronic device 1 according to an embodiment also narrows down the subspace and subspace bases including the heart sound from the reflected waves, and selects a plausible heart sound from the obtained waveform. Furthermore, the electronic device 1 according to an embodiment removes noise. In this way, the electronic device 1 according to an embodiment realizes the acquisition of a heart sound waveform. The electronic device 1 according to an embodiment can calculate the heartbeat interval by applying signal processing in an appropriate order.
[0103] The above-mentioned method will be described below. In order to accurately extract the heart sound of the subject 200, the electronic device 1 according to an embodiment performs appropriate signal processing on the signal (chirp signal) received by the electronic device 1 in an appropriate order. In this way, the electronic device 1 according to an embodiment narrows down the subspace and subspace base in which the signal component of the heart sound of the subject 200 exists. The electronic device 1 according to an embodiment first expresses the time series signal as a form of linear combination composed of a plurality of subspace bases. The electronic device 1 extracts the subspace having a large contribution rate to express the heart sound from the result. Then, the electronic device 1 reconstructs the time series signal using only the extracted subspace. By such processing, the electronic device 1 according to an embodiment can search for the subspace in which the heart sound of the subject 200 exists. To realize such a method, the electronic device 1 according to an embodiment may execute, for example, 2D-FFT processing of the reflected wave of the radar, other Fourier transform, wavelet transform, or processing by a group of derived methods thereof.
[0104] The electronic device 1 according to one embodiment may perform the following characteristic processing on a received signal in a step-by-step procedure. An outline of the characteristic processing by the electronic device 1 according to one embodiment is shown below. The electronic device 1 according to one embodiment may perform the following two characteristic processing. That is, the electronic device 1 according to one embodiment performs (1) subspace extraction processing, and (2) statistical signal processing for noise removal. Each processing will be described in more detail below.
[0105] (1) Subspace extraction process First, the electronic device 1 according to an embodiment executes a process for removing data considered to be unnecessary in a linear space from a received signal. The electronic device 1 according to an embodiment may execute a process for removing unnecessary subspace to leave necessary subspace. Specifically, the electronic device 1 may execute the following process.
[0106] (1-1) The electronic device 1 performs two Fourier transforms on the received chirp signal. The first Fourier transform makes it possible to obtain the distance between the object that reflected the transmitted wave and the electronic device 1. The second Fourier transform makes it possible to obtain the relative speed of the object that reflected the transmitted wave with respect to the electronic device 1. The electronic device 1 may apply a window function as appropriate to maintain the periodicity of the signal used in the Fourier transform. Furthermore, the electronic device 1 may extract only the micro-Doppler components including a point cloud where a subject 200 (e.g., a human or an animal) is present by estimating the direction of arrival of the reflected wave.
[0107] (1-2) The electronic device 1 executes a process for selecting a signal corresponding to vibration of the body surface from the set of multiple time-series signals (based on vibration) extracted in the process (1-1) above, on the micro-Doppler signal. The electronic device 1 executes a process using an inverse Fourier transform, an inverse wavelet transform, or a group of derived methods of these on the selected signal, thereby returning the selected signal to a time signal.
[0108] (1-3) The electronic device 1 extracts only the subspace that constitutes the heart sound from the time signal obtained by the process in (1-2) above using either a short-time Fourier transform, a wavelet transform, or a process based on a group of derived methods of these, or a band-pass filter.The electronic device 1 then reconstructs the time signal from the components of the extracted subspace (that constitutes the heart sound).At this point, the electronic device 1 can obtain a waveform that represents the heart sound.
[0109] (2) Statistical signal processing for noise reduction Next, in order to remove noise superimposed on the detected heart sound data, the electronic device 1 according to an embodiment may execute the following process.
[0110] (2-1) For example, when the electronic device 1 is installed in a moving object such as an automobile, the phonocardiogram waveform obtained by the above process (1-3) is likely to be superimposed with the body movement of the passenger and / or vibrations during driving. Therefore, the electronic device 1 generates a spectrogram or scalogram by performing frequency analysis using either a short-time Fourier transform or a wavelet transform on the phonocardiogram waveform obtained by the above process (1-3). Hereinafter, the spectrogram or scalogram may be collectively referred to as "time-frequency analysis information" as appropriate.
[0111] (2-2) The electronic device 1 approximates the time-frequency analysis information generated by the above process (2-1) to the sum of two sets of matrix products (each of which is a product of a frequency basis matrix and an activation matrix) by semi-supervised non-negative matrix factorization. Hereinafter, semi-supervised non-negative matrix factorization may be referred to as SSNMF (Semi-Supervised Non-negative Matrix Factorization) as appropriate. One of the two frequency basis matrices is a frequency basis matrix obtained in advance from a human heart sound waveform at rest. In the remaining three matrices, each matrix element may be updated by an update formula for minimizing a cost function determined by non-negative matrix factorization. Here, the update of the matrix elements may be repeated a predetermined number of times or until the cost function falls below a predetermined threshold. In addition, in non-negative matrix factorization, it is necessary to determine the number of basis vectors of the frequency basis matrix. Therefore, the electronic device 1 may determine the number of basis vectors of the frequency basis matrix to a plurality of values, and execute the processing by the SSNMF for the set number of bases.
[0112] (2-3) The electronic device 1 calculates time-frequency analysis information including a subspace expressing the heart sounds from the matrix product of a frequency basis matrix obtained in advance and an activation matrix corresponding to the frequency basis matrix (obtained in the process of (2-2) above). Then, the electronic device 1 combines the calculated time-frequency analysis information with the phase information of the time-frequency analysis information generated in the process of (2-1) above. Through such a process, the electronic device 1 can obtain a cardiac sound waveform from which noise components have been removed.
[0113] (2-4) The electronic device 1 calculates the degree of abnormality by Singular Spectrum Transformation (SST) for the waveform obtained by the above process (2-3) for the number of types of basis numbers set, and evaluates the presence or absence of noise. The heart sound waveform is basically a periodic signal unless it contains components other than heart sounds. Under normal circumstances, it is difficult to imagine that the heart beat interval will suddenly fluctuate in a short period of time. Therefore, if the degree of abnormality by SST increases, it is highly likely that noise is superimposed. Based on this principle, the electronic device 1 may obtain representative indicators such as the maximum value, median value, or weighted average of the degree of abnormality obtained from the waveform corresponding to each type of basis number, and may determine the basis number that minimizes these indicators as the best parameter. The electronic device 1 may use this parameter in the calculation of the heart beat interval, which is the next process of the obtained heart sound waveform.
[0114] Next, the operation of the electronic device 1 according to the embodiment will be described in more detail.
[0115] Fig. 12 is a flowchart showing an example of the operation performed by the electronic device 1 according to an embodiment. Fig. 13 is a flowchart showing in more detail an example of the operation of step S16 in Fig. 12. Hereinafter, the flow of the operation performed by the electronic device 1 according to an embodiment will be described with reference to Figs. 12 and 13.
[0116] Steps S11 to S14 shown in Fig. 12 can be performed in the same manner as the operations in steps S110 to S140 shown in Fig. 11. Furthermore, step S16 shown in Fig. 12 can be performed in the same manner as the operations in steps S150 and S160 shown in Fig. 11. Therefore, a detailed description of these operations will be omitted. The electronic device 1 according to an embodiment may remove noise from the vibration data extracted in step S14 shown in Fig. 12 (step S15), and calculate the RRI, HRV, and the like based on the data from which the noise has been removed (step S16).
[0117] The above-mentioned "(1) Subspace extraction process" generally describes the processes in steps S11 to S14 shown in FIG.
[0118] 12 may be performed in an environment where noises occur due to vibrations that occur when a moving body such as a car in which the electronic device 1 is installed is moving, and / or body movements of the passenger other than the heartbeat and breathing of the passenger. In such a case, the electronic device 1 can detect the passenger's heartbeat with good accuracy by removing the above-mentioned noises (step S15) and then performing a subsequent heartbeat interval calculation process (step S16).
[0119] In step S15, the signal processing unit 10 of the electronic device 1 may execute an iterative process using SSNMF and adjust hyperparameters using SST. More specifically, in step S15, the signal processing unit 10 may execute processes of steps S21 to S23 shown in Fig. 13. Hereinafter, the processes of steps S21 to S23 will be further described with reference to Fig. 13.
[0120] At the time when the process in step S14 in Fig. 12 is executed, the electronic device 1 can acquire a heartbeat waveform in which pulsations corresponding to R waves (narrow upward waves) are observed at regular intervals. However, as described above, depending on the environment in which the electronic device 1 performs detection, noise may be superimposed, which may affect the calculation accuracy of the RRI. Therefore, the electronic device 1 removes the noise superimposed on the heartbeat waveform by executing the process shown in step S15 in Fig. 12 (the process shown in steps S21 to S23 in Fig. 13).
[0121] When the operation shown in Fig. 13 starts, the signal processing unit 10 of the electronic device 1 first performs frequency analysis (step S21). In step S21, the signal processing unit 10 may convert the heart sound waveform into time-frequency analysis information by performing frequency analysis on the heart sound waveform. As described above, in an environment where the electronic device 1 installed inside a moving body such as a moving automobile detects the heartbeat of a passenger, noise is superimposed, so that the detected heart sound waveform is likely to be superimposed with body movement or vibration during driving. Therefore, in step S21, the signal processing unit 10 performs frequency analysis using either a short-time Fourier transform or a wavelet transform on the detected heart sound waveform to generate (calculate) a spectrogram or scalogram (time-frequency analysis information).
[0122] For example, the signal processing unit 10 may calculate the spectrogram according to the following equations (1) and (2).
number
number
[0123] In the above formula (1) and / or formula (2), Y represents a spectrogram, x(t) represents a finite-length time-series signal (cardiac sound waveform), and g(t) represents a window function. Furthermore, t (≧0) represents a time variable, N represents a frame of the Fourier transform, R represents a hop size, and ω represents an index of the discrete frequency. Furthermore, m represents a variable indicating the position of the time interval in which the Fourier transform is performed, and its numerical value is assumed to be within the range of [0, T-1]. Then, a new time variable is defined such that n=t-mR. Furthermore, the variable n varies within the range from 0 to the data length of the Fourier transform (0≦n≦N-1). x m represents the index of the windowed time frame of the m-th time series signal, and the hop size R is equal to the difference between the Fourier transform window length and the overlap length between each window.
[0124] The spectrogram Y that is finally generated (calculated) is expressed as the following equation (3), where the number of rows is the number of frequency indexes (bins) Ω (=N / 2), and the number of columns is the size of the number of frames T in the time direction.
number
[0125] After step S21, the signal processing unit 10 executes semi-supervised non-negative matrix factorization (SSNMF) (step S22). In step S22, the signal processing unit 10 may approximate the time-frequency analysis information (e.g., spectrogram Y) obtained in step S21 with a two-set matrix product by an update formula. That is, the signal processing unit 10 uses SSNMF to approximate the time-frequency analysis information calculated in step S21 to the sum of two sets of matrix products (each of which is a product of a frequency basis matrix and an activation matrix).
number
[0126] In the above formula (4), the matrix Y represents the time-frequency analysis information calculated in step S21 (for example, the spectrogram expressed as in the above formula (3)). The matrix F represents a frequency basis matrix with a basis number K obtained in advance from the cardiac sound waveform of a person at rest. The matrix F is a matrix as shown in the following formula (5).
number
[0127] Moreover, matrix G represents an activation matrix corresponding to matrix F. Matrix G is a matrix as shown in the following equation (6).
number
[0128] In addition, in the above formula (4), the matrix H represents a frequency basis matrix for configuring a waveform (that is, noise) other than the target signal with the basis number L. The matrix H is a matrix as shown in the following formula (7).
number
number
[0129] The signal processing unit 10 updates each matrix element of the three matrices (i.e., matrix G, matrix H, and matrix U) other than the above-mentioned matrix F, by an update formula for minimizing a cost function determined by non-negative matrix factorization. Here, several candidates for the cost function to be applied are considered. Two cost functions, Euclidean distance (Frobenius norm) and I-divergence (information divergence), are described below.
[0130] The cost function based on the Euclidean distance can be expressed as follows:
number
[0131] In the above formula (9), y ω,t represents the elements of matrix Y, and f ω,k represents the elements of matrix F, and g k,t represents the elements of matrix G, and h ω,l represents an element of the matrix H, and u l,t represents an element of the matrix U, and each element is a non-negative value.
[0132] The cost function based on I-divergence can be expressed as follows:
number
[0133] The minimization of the cost functions as shown in the above formulas (9) and (10) can be achieved by using the auxiliary function method.
[0134] For example, update equations corresponding to the cost function based on the Euclidean distance shown in the above equation (9) can be expressed as the following equations (11), (12), and (13).
number
number
number
[0135] Moreover, the update equations corresponding to the cost function based on I-divergence shown in the above equation (10) can be expressed as the following equations (14), (15), and (16).
number
number
number
[0136] The signal processing unit 10 may repeat the update of the matrix elements a predetermined number of times or until the cost function falls below a predetermined threshold. In addition, in the non-negative matrix factorization, it is necessary to determine the number L of basis vectors of the frequency basis matrix. Therefore, the signal processing unit 10 may determine the number of basis vectors of the frequency basis matrix to a plurality of numbers and execute the processing by the SSNMF as many times as the set number of bases. Next, the signal processing unit 10 converts the matrix FG (time-frequency analysis information) that has been processed by the SSNMF back into a time signal by using the phase information of the matrix Y, by the inverse short-time Fourier transform, the inverse wavelet transform, or any of the derived methods based on these.
[0137] For example, when using the inverse short-time Fourier transform, the signal processing unit 10 may perform the calculation according to the following equations (17) to (20).
number
number
number
number
[0138] In the above formulas (17) to (20), R indicates the hop size between successive DFTs. Also, Y'(ω,m) indicates a specific element present in the spectrogram matrix obtained by SSNMF. The number of bins of the discrete frequencies of Y' is Ω, but when performing the inverse Fourier transform, it is treated as having N types, which is twice Ω. This is because the power spectrum when performing the Fourier transform on a real signal becomes a symmetrical value when the intermediate frequency is used as an axis, and when deriving a spectrogram, the indexes of the 0th to N / 2th discrete frequencies are stored as information.
[0139] The signal processing unit 10 may apply an inverse DFT to the result of the discrete Fourier transformation (DFT) obtained for each frame to obtain the time signal shown on the left side of the above formula (20).The signal processing unit 10 may then perform overlap-add of all these signals to calculate a reconstructed heart sound waveform (shown on the left side of the above formula (18)).Here, the domain of the time variable t is from 0 to the length of the time series signal.
[0140] If the above-described processing is applied to the matrix HU instead of the matrix FG, the waveform that was interpreted as noise will be reconstructed.
[0141] Fig. 14 is a diagram showing an example of a series of processing procedures mainly based on the above-mentioned SSNMF summarized as pseudocode. Also, Fig. 15 is a diagram showing the series of processing procedures summarized using a cardiac sound waveform and a spectrogram.
[0142] Fig. 15(a), Fig. 15(b), and Fig. 15(c) shown in the upper part of Fig. 15 respectively show examples of time signals in the time domain, and Fig. 15(d), Fig. 15(e), and Fig. 15(f) shown in the lower part of Fig. 15 respectively show examples of spectrograms in the time-frequency domain.
[0143] The time signal shown in FIG. 15(a) indicates a signal (original signal) before processing to remove noise. The signal processing unit 10 can obtain a spectrogram Y shown in FIG. 15(d) by applying a short-time Fourier transform to the time signal shown in FIG. 15(a). Hereinafter, the short-time Fourier transform is appropriately referred to as STFT (short-time Fourier transform or short-term Fourier transform). As described above, the signal processing unit 10 approximates the spectrogram Y shown in FIG. 15(d) by the sum of a matrix FG shown in FIG. 15(e) and a matrix HU shown in FIG. 15(f). The signal processing unit 10 can obtain a time signal shown in FIG. 15(b) by applying an inverse short-time Fourier transform (inverse STFT) to the matrix FG shown in FIG. 15(e). The time signal shown in FIG. 15(b) indicates a signal (target signal) after processing to remove noise. In this way, the signal processing unit 10 can obtain the target signal (FIG. 15(b)) from which noise has been removed by performing the process of converting the matrix FG shown in FIG. 15(e) into a time signal. On the other hand, as described above, by performing the process of converting the matrix HU shown in FIG. 15(f) into a time signal, a time signal (FIG. 15(c)) in which the waveform interpreted as noise is reconstructed can be obtained.
[0144] As described above, the signal processing unit 10 converts the original waveform into a raw The signal processor 10 converts the signal (FIG. 15(a)) into a spectrogram (FIG. 15(d)), and generates matrices composed of random values (non-negative values) of a predetermined size (FIG. 15(e) and (FIG. 15(f)). The signal processor 10 sets the matrices thus generated as the initial values of the three matrices, matrix G, matrix H, and matrix U. Next, the signal processor 10 calculates the update formula Func G , Func H , and Func U The signal processing unit 10 updates the elements of the three matrices G, H, and U by using the above formulas (11) to (16). The signal processing unit 10 repeats this updating process a predetermined number of times (for example, IterNum times). In this way, the signal processing unit 10 finally updates the three matrices G [IterNum] , H [IterNum] , U [IterNum]In one embodiment, if the cost function D of Y and FG+HU (the above formulas (9) and (10)) falls below a threshold ε during the above-mentioned repetition, the signal processing unit 10 may abort the update process. Finally, the signal processing unit 10 converts the spectrogram (FIG. 15(e)) into a time signal waveform with noise removed by inverse short-time Fourier transform. denoise (Figure 15(b)).
[0145] After step S22, the signal processing unit 10 executes a process of selecting the best waveform by the singular spectrum transform (SST) method (step S23). In step S23, the signal processing unit 10 may select the best waveform by calculating the degree of abnormality of the waveform by the SST and updating the optimal basis number. The signal processing unit 10 may adjust the hyperparameters by the SST based on the result obtained in step S22.
[0146] In step S23, the signal processor 10 may evaluate the presence or absence of noise by deriving the degree of abnormality by SST for the number of types of basis number L. As described above, the heart sound waveform is basically a periodic signal unless it contains components other than heart sounds. Under normal circumstances, it is difficult to imagine that a sudden change in the heartbeat interval occurs in a short period of time. Therefore, if the degree of abnormality by SST increases, it is highly likely that noise is superimposed. SST extracts two adjacent sections from the entire time series signal, creates two matrices (referred to as a history matrix H1 and a check matrix H2, respectively) from each time series signal, and applies a multivariate analysis technique. SST allows quantitative evaluation of the degree of deviation of the waveforms in the two sections.
[0147] Fig. 16 is a diagram showing an example of anomaly detection using SST. Fig. 16(a) shown in the upper part of Fig. 16 shows the input waveform. Fig. 16(b) shown in the lower part of Fig. 16 shows the degree of anomaly obtained by processing. As shown in Fig. 16(b), it can be confirmed that, according to SST, the degree of anomaly increases at the same time as the trend of the waveform changes.
[0148] The processing procedure using SST is explained below. As a multivariate analysis method, mainly either the subspace method or the singular value decomposition can be used. Here, the processing when the singular value decomposition is used is explained.
[0149] In SST, first, when the entire time series signal is expressed as shown in the following equation (21) (corresponding to the above equation (18)), the partial time series can be defined as shown in the following equation (22).
number
number
[0150] In the above formula (22), T means transposition. Also, assume that each sample point of the time series signal is recorded at a certain time interval. In this case, the history matrix H1 and the check matrix H2 including n partial time series signals at time t can be defined as the following formulas (23) and (24).
number
number
[0151] Here, γ is an integer that determines the distance between the two intervals to be compared. It is assumed that the column vectors constituting the matrices H1 and H2 contain pattern information that appears in the time series data. The signal processing unit 10 then performs singular value decomposition (SVD) on these matrices to discover feature patterns from the n column vectors.
[0152] The signal processing unit 10 selects a matrix U1 including r left singular vectors with particularly large singular values from the left singular vector matrices U1 and U2 of the matrices H1 and H2 obtained by such processing. (r) and U2 (r)The matrix U1 generated in this way is (r) and U2 (r) From this, the signal processing unit 10 can calculate the degree of abnormality a(t) according to the following equations (25) to (27).
number
number
number
[0153] The degree of anomaly a(t) defined as in the above formula (27) takes a value between 0 and 1. When the difference between the two time series is large, the degree of anomaly a(t) approaches 1. When the similarity between the two time series is high, the degree of anomaly a(t) approaches 0. The hyperparameters used in this method include the length ω of the partial time series, the number n of column vectors, the interval γ between two intervals, and the number r of left singular vectors to be extracted. Depending on the combination of these hyperparameters, the sensitivity and / or stability of anomaly detection varies. Therefore, in anomaly detection using SST, it is necessary to adjust the hyperparameters in advance. When anomaly detection using SST is applied to a cardiac sound waveform, the hyperparameters may be set so that several cycles of heartbeats are included in the partial time series. In this way, the stability of change detection can be maintained.
[0154] The signal processing unit 10 obtains representative indices such as the maximum value, median value, or weighted average of the degree of abnormality a(t) obtained from the waveform corresponding to each basis number L, and selects the basis number L that has the smallest value. best may be set as the best parameter. The signal processor 10 may then use the cardiac sound waveform thus obtained in the next process, which is calculation of the cardiac interval. Similarly, in the next frame loop, the signal processor 10 may calculate the basis number Lbest and the adjacent basis number [L best -num base ,…,L best ,…,L best +num base] may be used as a parameter for noise removal processing by SSNMF in step S22. In this way, the signal processor 10 can stabilize the extraction result of the cardiac sound waveform. Then, the signal processor 10 adjusts the basis number L based on the degree of abnormality obtained by the SST in the next process, step S23. best In this manner, while repeating the process, the signal processing unit 10 can optimize the parameters used for noise removal based on the index of abnormality degree.
[0155] FIG. 17 is a diagram showing an example of a series of processing steps as described above summarized as pseudo code. In FIG. base is a predetermined constant. base The signal processor 10 repeats the SSNMF process according to the value of a(t). The signal processor 10 derives the degree of anomaly a(t) by SST for the waveform obtained by the iterative process, and derives a representative value θ such as the average value, minimum value, or median value. Here, the maximum value is derived as the representative value θ. After the iterative process, the signal processing unit 10 determines the basis number for which the representative value θ is smallest as the basis number L best The signal processing unit 10 updates the waveform to be selected for calculating the RRI. denoise According to the update result, the electronic device 1 can sequentially optimize the basis number by such a process.
[0156] For example, num base Assume that the basis number L that results in the lowest abnormality is set to 5. In this case, the signal processing unit 10 sets the basis number L to be used in the SSNMF to five different consecutive integers (for example, 18, 19, 20, 21, and 22), and performs SSNMF five times with each basis number. Then, the signal processing unit 10 derives the degree of abnormality by SST five times. Here, the basis number L that results in the lowest abnormality is best Assume that the base number L is 22. In this case, in the next loop process, the signal processing unit 10 repeats the processes of SSNMF and SST with the basis numbers set to 20, 21, 22, 23, and 24. Then, the signal processing unit 10 bestThe heartbeat interval may be derived using the heart sound waveform extracted using (=22).
[0157] Next, a description will be given of a usage mode when operating the electronic device 1 according to an embodiment. Hereinafter, a description will be given of the implementation as hardware and the process flow when operating the electronic device 1 according to an embodiment.
[0158] As shown in FIG. 2, in the electronic device 1 according to an embodiment, an audiovisual device 70 may be connected to the communication interface 50 in a wired and / or wireless manner. The audiovisual device 70 may be any device capable of making a predetermined notification by sound and / or video to a passenger, such as a driver, who is riding in a moving body, such as a car, in which the electronic device 1 is installed. In addition, the audiovisual device 70 is not necessarily limited to a device that makes a predetermined notification by sound and / or video, and may be any device that makes a predetermined notification by, for example, tactile feedback. In FIG. 2, the audiovisual device 70 is connected to the communication interface 50, but may be connected to, for example, the signal processing unit 10 and / or the external device 60 in a wired and / or wireless manner.
[0159] Furthermore, the calculation unit 14 of the signal processing unit 10 shown in Fig. 2 may further calculate a CVRR (Coefficient of Variation of RR interval) in addition to the processing already described in Fig. 2. Here, CVRR may mean the coefficient of variation of RR interval variation. CVRR is an index showing the degree of variation of RRI in a certain period of time as a percentage. It has been reported that CVRR is correlated with the activity of the subject's parasympathetic nerves, and it is expected to be used to quantify the subject's fatigue.
[0160] The electronic device 1 transmits the biological information obtained by calculating the CVRR in the calculation unit 14 to the audiovisual device 70 via the communication interface 50. This allows the subject to understand the processing result by the electronic device 1. In addition, when the subject is driving a moving object such as a car in which the electronic device 1 is installed and there is a high possibility of a danger due to drowsiness or the like occurring, the electronic device 1 can output visual information and / or audio information from the audiovisual device 70 to the subject. The visual information output by the audiovisual device 70 may be, for example, a rapid flashing of a lamp or the like. In addition, the visual information output by the audiovisual device 70 may be, for example, a warning sound such as a buzzer sound.
[0161] Fig. 18 is a flowchart for explaining an example of operation of the electronic device 1 according to an embodiment as a driver monitoring system (DMS). Fig. 18 may show a processing procedure for updating a frequency basis matrix used in the SSNMF by the electronic device 1 according to an embodiment.
[0162] In the operation shown in Fig. 18, the electronic device 1 according to an embodiment needs a heart sound waveform that does not include noise. Therefore, in the operation shown in Fig. 18, the electronic device 1 needs to acquire the heart sound at a timing when no vibration occurs in a moving object such as a car in which the electronic device 1 is installed.
[0163] 19 starts, the signal processing unit 10 acquires the traveling speed of the moving object on which the electronic device 1 is installed (step S31). In step S31, the signal processing unit 10 may acquire the vehicle speed of the moving object based on the detection result by an inertial measurement unit (IMU), the detection result of the rotation speed of the tires, and / or the detection result by various devices such as an on-board camera.
[0164] In step S32, the signal processing unit 10 determines whether or not the moving object on which the electronic device 1 is installed has been stopped for a certain period of time.
[0165] As shown in step S33, when no movement is observed for a certain period of time that would result in superimposition of noise associated with the movement of a moving object, the signal processing unit 10 may acquire a cardiac sound waveform for the certain period of time.
[0166] Next, as shown in step S34, the signal processing unit 10 determines whether or not the moving object in which the electronic device 1 is installed is moving. If it is determined that the moving object is moving, the signal processing unit 10 determines whether or not a section in which the SST abnormality value is low exists for a predetermined period of time, as shown in step S35. The signal processing unit 10 may perform change detection by SST on the phonocardiogram waveform obtained from when the moving object stops until it resumes running. Furthermore, if a section in which the SST abnormality level is below a certain threshold exists for a predetermined period of time or more (Yes in step S35), the signal processing unit 10 uses the phonocardiogram waveform of that section to create a basis matrix.
[0167] Next, as shown in step S36, the signal processing unit 10 transforms the time-series signal into the time-frequency domain, and approximates the time-frequency analysis information by non-negative matrix factorization (NMF).
[0168] Next, as shown in step S37, the signal processing unit 10 may update the basis matrix used for removing noise, and then end the operation shown in FIG.
[0169] Fig. 19 is a diagram showing an example of abnormality detection performed on a heart sound waveform acquired by an electronic device 1. Fig. 19 shows an example in which an electronic device 1 installed inside a moving body such as an automobile detects a heart sound waveform of an occupant of the moving body. Fig. 19(a) shown in the upper part of Fig. 19 shows an input waveform. Fig. 19(b) shown in the lower part of Fig. 19 shows the degree of abnormality of the input waveform.
[0170] As shown in FIG. 19, the mobile object in which the electronic device 1 is installed stops at a time of about 20 seconds and resumes traveling at a time of about 140 seconds. In the example shown in FIG. 19, if the threshold of the degree of anomaly shown in FIG. 19(b) is 0.075 and the time length required for the basis matrix is 40 seconds, the section α shown in FIG. 19(b) is below the threshold of the degree of anomaly for a length of 40 seconds or more. In this case, the signal processing unit 10 updates the basis matrix used for noise elimination. On the other hand, the period in which the period of the period β shown in FIG. 19(b) is below the threshold of the degree of anomaly is less than 40 seconds. In this case, the signal processing unit 10 does not update the basis matrix used for noise elimination.
[0171] Also, if the section β shown in FIG. 19(b) satisfies the conditions of the degree of abnormality and the time length, this is the latest data. Therefore, the signal processing unit 10 may update the basis matrix used for noise elimination. The signal processing unit 10 uses the history matrix H1 and the check matrix H2 to obtain the degree of abnormality at the start point and the end point of the section adopted for updating the basis matrix used for noise elimination. The signal processing unit 10 may select one point from each time series signal from any sample point included in the two time series used for generating the history matrix H1 and the check matrix H2 so that a waveform of the length of the section of the degree of abnormality and the same time length is obtained.
[0172] The signal processing unit 10 converts the cardiac sound waveform into time-frequency analysis information such as a spectrogram, using the two selected sample points as the start and end points, and updates the basis matrix F by NMF, as shown in the following equation (28).
number
[0173] In the above formula (28), Y target indicates a spectrogram expressed as the following equation (29).
number
[0174] In the above formula (28), F represents a basis matrix expressed as in the following formula (30).
number
[0175] In the above equation (28), Q represents an activation matrix expressed as in the following equation (31).
number
[0176] The signal processing unit 10 performs approximation of the matrices shown in the above formulas (29) to (31) by the update formula shown in the above formula (28). Two basic update formulas will be described below.
[0177] First, the update equations corresponding to the cost function based on the Euclidean distance can be expressed as the following equations (32) and (33).
number
number
[0178] Moreover, the update equations corresponding to the cost function based on I-divergence can be expressed as the following equations (34) and (35).
number
number
[0179] The signal processing unit 10 selects one of the above update equations and repeats the update a fixed number of times, or performs approximation until a fixed distance is reached.
[0180] Fig. 20 is a diagram showing an example of a matrix approximation result by NMF in accordance with the above formula (28) when a spectrogram of a cardiac sound waveform is input to the signal processing unit 10. In Fig. 20, K represents the basis number.
[0181] Fig. 21 is a flowchart for explaining an example of operation of the electronic device 1 according to an embodiment as a driver monitoring system (DMS). Fig. 21 is a diagram for explaining an operation of the electronic device 1 according to an embodiment to monitor a driver's biological information and issue an alarm in a predetermined case. In Fig. 21, the processing of steps S41 to S44 may correspond to steps S11 to S16 shown in Fig. 12. In addition, in step S44, the signal processing unit 10 may calculate at least one of BPM (beats per minute), HRV, and CVRR in addition to the heart rate interval (RRI).
[0182] Next, in step 45, the signal processing unit 10 presents (displays) information based on the detected biosignal on an audiovisual device 70 such as a display unit. As described above, the signal processing unit 10 may present the information based on the detected biosignal from the audiovisual device 70 as auditory information and / or tactile information.
[0183] Next, in step S46, the signal processing unit 10 may determine whether or not there is an abnormality in the subject's health condition based on the detected biosignal. When it is estimated based on the detected biosignal that there is a high possibility that a dangerous situation will occur in the subject's health condition, the signal processing unit 10 may output a buzzer sound and / or a rapid flashing of a lamp from the audiovisual device 70 to leave the subject in potential danger (step S47).
[0184] Thus, the electronic device 1 according to an embodiment includes a signal processor 10 that detects a monitoring target based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the monitoring target. The signal processor 10 generates first time-frequency analysis information Y1 based on the transmission wave and the reflected wave. The signal processor 10 calculates the first time-frequency analysis information Y1 by the SSNMF as Y1=FG+HU using frequency basis matrices F and H and activation matrices G and U, and calculates an activation matrix G using a predetermined cost function with the frequency basis matrix F being a frequency basis matrix obtained from a reference heart sound waveform obtained in advance. Then, the signal processor 10 generates a heart sound waveform of the monitoring target based on the product FG of the frequency basis matrix F and the activation matrix G.
[0185] Furthermore, in the electronic device 1 according to an embodiment, when the number of basis vectors of the frequency basis matrices F and H is L, the signal processing unit 10 may execute the following process. That is, for a heart sound waveform generated based on the product FG of the frequency basis matrix F and the activation matrix G, the signal processing unit 10 may regenerate the heart sound waveform using the number of basis vectors L when the degree of abnormality α obtained by the singular spectrum transform method is minimized. In this case, the degree of abnormality α(t) may be defined as in the above formula (27).
[0186] Furthermore, in the electronic device 1 according to an embodiment, the signal processing unit 10 may generate second time-frequency analysis information Y2 for the occupant's heart sound waveform when the motion of the moving object on which the occupant is riding is equal to or less than a predetermined value. In this case, the signal processing unit 10 may update the basis matrix F by expressing the second time-frequency analysis information Y2 as the product of the basis matrix F and the activation matrix Q, Y2=FQ, using a cost function by NMF.
[0187] In addition, in the electronic device 1 according to an embodiment, when the speed of the moving object on which the passenger is riding is equal to or lower than a predetermined value, the movement of the moving object on which the passenger is riding may be equal to or lower than a predetermined value. The passenger may be the same person as the monitoring target. Furthermore, the electronic device 1 may be installed inside the moving object.
[0188] As described above, the electronic device 1 according to one embodiment can measure the vibrations of the subject's body surface by applying signal processing to the reflected waves (received waves) obtained from the radar. As a result, the electronic device 1 according to one embodiment can obtain a good cardiac sound waveform of a human or animal. Furthermore, the electronic device 1 according to one embodiment can detect the presence or absence of noise, including road noise, superimposed on the obtained cardiac sound waveform by multivariate analysis. And, the electronic device 1 according to one embodiment can separate the heart sound from the noise when the obtained cardiac sound waveform contains noise.
[0189] Fig. 22 is a diagram showing an example of a heart sound waveform in which noise has been removed by the electronic device 1 according to an embodiment. Fig. 22(a) shown in the upper part of Fig. 22 is a diagram showing an example of a heart sound waveform (for 20 seconds) before processing to remove noise is performed by the electronic device 1 according to an embodiment. Fig. 22(b) shown in the lower part of Fig. 22 is a diagram showing an example of a heart sound waveform (for 20 seconds) after processing to remove noise is performed by the electronic device 1 according to an embodiment.
[0190] As shown in Fig. 22(a), in the left half (first 10 seconds) of the heart sound waveform before noise removal processing, the heart sound peak is hidden by road noise. According to the electronic device 1 according to one embodiment, the heart sound waveform is converted into frequency information, and then only the target signal components are extracted, thereby realizing noise removal as shown in the left half (first 10 seconds) of Fig. 22(b).
[0191] Moreover, the electronic device 1 according to the embodiment does not require a huge amount of data, as in deep learning, to remove road noise. The electronic device 1 according to the embodiment can remove noise such as road noise if the cardiac waveform of an individual at rest is obtained for, for example, several tens of seconds.
[0192] Furthermore, according to the electronic device 1 of an embodiment, by performing SST as shown in FIG. 19, for example, it is possible to detect noise superimposed on the original waveform shown in FIG. 19(a) as shown in FIG. 19(b). For example, the electronic device 1 of an embodiment can determine that there is a high possibility that noise is included when the degree of abnormality based on SST is high. According to the electronic device 1 of an embodiment, it is possible to select a heart sound waveform for teacher data used for road noise removal and / or readjust optimal hyperparameters based on the degree of abnormality obtained by performing SST.
[0193] (Other embodiments) Other embodiments will be described below.
[0194] The electronic device 1 according to the above-described embodiment is assumed to remove noise from a heart sound waveform acquired by, for example, a millimeter wave radar. However, the electronic device 1 according to one embodiment finally realizes estimation of the heart beat interval by capturing the vibration of the pulsation of humans and animals that is transmitted to the body surface. Therefore, the electronic device 1 according to another embodiment can also remove noise from a waveform obtained from a device that can measure the vibration of the installation position, such as an acceleration sensor.
[0195] The electronic device 1 according to another embodiment can remove noise not only from the waveform of the heartbeat of a human or animal, but also from the respiratory waveform of a human or animal. Respiration causes a larger displacement fluctuation on the body surface than the heartbeat, and the displacement fluctuation tends to be significant. Therefore, the electronic device 1 according to another embodiment can obtain the respiratory waveform of a human or animal more easily than the waveform of the heartbeat. In addition, the electronic device 1 according to the embodiment of the present disclosure can remove noise from the body movements of the head, eyelids, eyeballs, pupils, feet, hands, and / or mouth of a human or animal.
[0196] As described above, the electronic device 1 according to one embodiment may be installed in any mobile object moving on a public road, for example. However, the electronic device 1 according to another embodiment may be similarly applied not only to mobile objects moving on a public road such as automobiles, but also to environments where noise may be generated, such as airplanes, ships, or trains. The electronic device 1 according to another embodiment may also be similarly applied to noise generated by the body movement of a subject in an indoor environment. Furthermore, the electronic device 1 according to one embodiment may be applied to measuring the heart rate and / or removing noise of a subject riding on, for example, a motorcycle, a bus, a bicycle, a truck, a construction vehicle, or a tractor.
[0197] In another embodiment, the method of evaluating noise removal performance according to the degree of abnormality performed in step S23 of Fig. 13 may be applied to select a heart sound waveform with less noise when acquiring a heart sound waveform performed in step S33 of Fig. 18. This process may be important when acquiring an appropriate frequency basis matrix in the SSNMF used for noise removal.
[0198] In another embodiment, the method of evaluating the noise removal performance according to the degree of anomaly performed in step S23 in Fig. 13 can also be applied to the case where the SSNMF process is omitted when the level of noise is relatively low in the process performed in step S43 in Fig. 21. By omitting the SSNMF process in this way, the load of the calculation process can be reduced. Also, in the process performed in step S43 in Fig. 21, if the noise is relatively low, the noise may be removed by a simple NMF instead of the SSNMF.
[0199] In another embodiment, the update formula used in the SSNMF process performed in step S22 in Fig. 13 is not limited to the above, and another effective update formula may be used. For example, the electronic device 1 according to another embodiment may use an update formula with a constraint that two frequency basis matrices are mutually orthogonal, as shown in the following formula (36).
number
[0200] Furthermore, for example, the electronic device 1 according to another embodiment may use an update equation with a constraint that maximizes the KL (Kullback-Leibler) distance between two frequency basis matrices, as shown in the following equation (37).
number
[0201] The electronic device 1 according to the other embodiment can perform signal separation with higher accuracy by using an update equation that takes into account constraints such as those in the above equations (36) and (37).
[0202] In the above-described embodiment, the observation matrix applied to the SSNMF and NMF is, for example, a spectrogram. However, in other embodiments, time-frequency analysis information obtained by wavelet transform and its derived methods may be used instead of the spectrogram.
[0203] In another embodiment, when the computational load of detecting abnormalities in a time-series signal using SST makes real-time processing difficult, the processing speed may be improved by downsampling to a degree that does not distort the cardiac sound waveform.
[0204] 2, the signal processing unit 10 has been described as including functional units such as the heartbeat extraction unit 13 and the calculation unit 14. However, in one embodiment, the processing performed by the heartbeat extraction unit 13 and / or the calculation unit 14 may be performed by an external computer or processor.
[0205] As described above, the electronic device 1 according to an embodiment detects weak vibrations such as heartbeats using, for example, a millimeter wave sensor having a plurality of transmitting antennas and a plurality of receiving antennas. When the electronic device 1 according to an embodiment does not detect a target, it detects the body movement of the target while changing the transmission phase of the antenna to perform the beamforming pattern of the transmitting antenna. On the other hand, when the electronic device 1 according to an embodiment detects the body movement of the target, it performs beamforming in the direction of the body movement and detects the heartbeat. In this way, the electronic device 1 according to an embodiment can improve signal quality by automatically detecting the direction of the human body. Therefore, the electronic device 1 according to an embodiment can improve the detection accuracy and / or detection range of the heartbeat. Therefore, the electronic device 1 according to an embodiment can detect the human heartbeat with high accuracy.
[0206] Although the present disclosure has been described based on the drawings and examples, it should be noted that a person skilled in the art can easily make various modifications or corrections based on the present disclosure. Therefore, it should be noted that these modifications or corrections are included in the scope of the present disclosure. For example, the functions included in each functional unit can be rearranged so as not to be logically inconsistent. Multiple functional units, etc. may be combined into one or divided. Each embodiment of the present disclosure described above is not limited to being implemented faithfully to each of the embodiments described, and may be implemented by combining each feature as appropriate or omitting a part. In other words, the contents of the present disclosure can be modified and corrected in various ways by a person skilled in the art based on the present disclosure. Therefore, these modifications and corrections are included in the scope of the present disclosure. For example, in each embodiment, each functional unit, each means, each step, etc. can be added to other embodiments so as not to be logically inconsistent, or replaced with each functional unit, each means, each step, etc. of other embodiments. In addition, in each embodiment, multiple functional units, each means, each step, etc. can be combined into one or divided. Furthermore, each of the above-described embodiments of the present disclosure is not limited to being implemented faithfully according to each of the described embodiments, but may be implemented by combining each feature or omitting some features as appropriate.
[0207] The above-described embodiment is not limited to being implemented only as the electronic device 1. For example, the above-described embodiment may be implemented as a control method for an apparatus such as the electronic device 1. Furthermore, the above-described embodiment may be implemented as a program executed by an apparatus such as the electronic device 1, or as a storage medium or recording medium on which a program is recorded.
[0208] Moreover, the electronic device 1 according to the above-mentioned embodiment has been described as including components constituting a so-called radar sensor, such as the transmitting antenna array 24 and the receiving antenna array 31. However, the electronic device according to one embodiment may be implemented as, for example, a configuration such as a signal processing unit 10. In this case, the signal processing unit 10 may be implemented as having a function of processing signals handled by, for example, the transmitting antenna array 24 and the receiving antenna array 31. [Explanation of symbols]
[0209] 1 Electronic equipment 10 Signal Processing Section 11 Signal generation processing section 12 Receiving signal processing section 13 Heart Rate Extraction Unit 14 Calculation section 21 Transmit DAC 22 Transmitting circuit 23 Millimeter wave transmission circuit 24 Transmitting Antenna Array 31 Receiving Antenna Array 32 Mixer 33 Receiving circuit 34 Receive ADC 50 Communication Interface 60 External equipment 70 Audio-visual equipment
Claims
1. An electronic device including a signal processing unit that detects a monitoring target based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the monitoring target, The signal processing unit includes: The time-frequency analysis information calculated based on the transmitted wave and the reflected wave is subjected to non-negative matrix factorization using basis information of a cardiac acoustocardiogram obtained in advance. and an electronic device that generates a cardiac waveform to be monitored.
2. An electronic device including a signal processing unit that detects a monitoring target based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the monitoring target, The signal processing unit includes: generating first time-frequency analysis information Y1 based on the transmitted wave and the reflected wave; Semi-supervised non-negative matrix factorization The first time-frequency analysis information Y1 is calculated as Y1=FG+HU using frequency basis matrices F and H and activation matrices G and U, and the frequency basis matrix F is set as a frequency basis matrix obtained from a reference heart sound waveform obtained in advance, and the activation matrix G is calculated using a predetermined cost function; An electronic device that generates a heart sound waveform of the monitored subject based on a product FG of the frequency basis matrix F and the activation matrix G.
3. The signal processing unit includes: When the number of basis vectors of the frequency basis matrices F and H is L, Regarding the heart sound waveform generated based on the product FG of the frequency basis matrix F and the activation matrix G, The degree of anomaly α(t) by the singular spectrum transform method is expressed as follows: Define it as The electronic device according to claim 2 , wherein the heart sound waveform is regenerated using the number L of basis vectors when the degree of abnormality α(t) is minimized.
4. The signal processing unit includes: generating second time-frequency analysis information Y2 for the phonocardiogram of the passenger when the motion of the vehicle on which the passenger is riding is equal to or less than a predetermined value; 3. The electronic device according to claim 2, wherein the basis matrix F is updated by expressing the second time-frequency analysis information Y2 as a product of the basis matrix F and an activation matrix Q, Y2=FQ, using a cost function by non-negative matrix factorization.
5. The electronic device according to claim 4 , wherein, when the speed of the moving object is equal to or lower than a predetermined value, the movement of the moving object on which the passenger is riding is determined to be equal to or lower than a predetermined value.
6. The electronic device according to claim 4 , wherein the occupant is the same person as the monitored person.
7. The electronic device according to claim 4 , which is installed inside the moving body.
8. detecting the monitoring target based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the monitoring target; generating first time-frequency analysis information Y1 based on the transmitted wave and the reflected wave; Semi-supervised non-negative matrix factorization A step of calculating the activation matrix G using a predetermined cost function by using the first time-frequency analysis information Y1 as Y1=FG+HU using frequency basis matrices F and H and activation matrices G and U, and setting the frequency basis matrix F as a frequency basis matrix obtained from a reference heart sound waveform obtained in advance; generating a heart sound waveform of the monitored object based on a product FG of the frequency basis matrix F and the activation matrix G; A method for controlling an electronic device, comprising:
9. For electronic devices, detecting the monitoring target based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the monitoring target; generating first time-frequency analysis information Y1 based on the transmitted wave and the reflected wave; Semi-supervised non-negative matrix factorization A step of calculating the activation matrix G using a predetermined cost function by using the first time-frequency analysis information Y1 as Y1=FG+HU using frequency basis matrices F and H and activation matrices G and U, and setting the frequency basis matrix F as a frequency basis matrix obtained from a reference heart sound waveform obtained in advance; generating a heart sound waveform of the monitored object based on a product FG of the frequency basis matrix F and the activation matrix G; A program to execute.
Citation Information
Patent Citations
Signal processing method and sound signal processor
JP2017151228A
Methods and systems for improved signal decomposition
US20190348059A1
Noise elimination device, noise elimination method, and position specification method
WO2016051565A1
Discrimination device, learning method for discrimination device, and computer program
WO2021166243A1
System and Method for Mitigation of Random Body Movement and Interference Effects in Radar-Based Vital Signs Monitoring Systems in Medical Applications
WO2023016948A1
