Non-contact electrocardiogram monitoring method and system based on millimeter-wave sensing

Abstract

The present application provides a non-contact electrocardiogram monitoring method and system based on millimeter-wave sensing. The method includes: non-contact collecting the current original signal of the measured target from the thoracic region of the measured target in a millimeter-wave sensing manner; performing frequency-domain filtering on the original signal based on beamforming to extract the complex signal corresponding to the cardiac region of the measured target; inputting the complex signal into a preset electrocardiogram cross-modal mapping model so that the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the measured target. The present application can avoid losing key information of cardiac activities such as the electromagnetic wave frequency offset and amplitude change that may be caused by the loss of cardiac electrical activities, can effectively improve the comprehensiveness and reliability of the focus on cardiac activities, can effectively improve the extraction accuracy of cardiac activity information, and thus can effectively improve the reliability and accuracy of non-contact electrocardiogram monitoring.

Description

Technical Field

[0001] This application relates to the technical field of electrocardiogram monitoring, and in particular to a non-contact electrocardiogram monitoring method and system based on millimeter-wave sensing. Background Art

[0002] Millimeter-wave radar, as a non-contact sensing device, has received wide attention in the field of health monitoring in recent years, and it has the potential ability to continuously and unobtrusively monitor electrocardiograms. The activities of the heart can be divided into electrical activities and mechanical activities, where the mechanical activities are caused by the conduction of electrical activities and myocardial excitation. Both of these activities can be monitored by the radar emitting electromagnetic waves. The electromagnetic waves emitted by the transmitting antenna will be reflected back to the receiving antenna of the radar by the chest cavity. The reflected signal carries information about the heart activities, which can be extracted through a series of algorithms and modeled as electrocardiogram signals. Among them, the electrical activities of the heart will generate an electromagnetic field, which causes the frequency of the electromagnetic waves to shift during transmission, and the mechanical activities during the heartbeat process cause minute displacement fluctuations in the chest cavity, which will cause regular changes in the phase of the electromagnetic waves.

[0003] Currently, existing non-contact cardiac monitoring technologies can be divided into two categories: heart rate monitoring and electrocardiogram monitoring. Among them, heart rate monitoring belongs to early work, and the average heart rate within a certain period of time can be obtained through signal spectrum processing methods. However, due to the limitations of algorithms and sensors, these methods can only estimate the approximate heart rate and cannot extract the minute cardiac events within a single heartbeat. For the second category, non-contact monitoring of electrocardiograms is mainly through signal processing of millimeter-wave radar and cross-domain mapping of heart activities, that is, mapping from the mechanical activities of the chest cavity to the electrical activities of the heart to construct an electrocardiogram.

[0004] However, the existing non-contact electrocardiogram monitoring method based on millimeter-wave sensing focuses on the heart mechanical activity area according to fixed rule spectral features, and the focusing and extraction accuracy of this method for heart activities are insufficient. In addition, the existing method only extracts the mechanical activity information of the heart in the radar reflection signal and ignores the possible frequency shift of electromagnetic waves caused by heart electrical activities. The above two factors result in insufficient consistency between the electrocardiogram monitoring waveform and the gold standard device during the implementation of electrocardiogram monitoring in the existing technology, and the accuracy of the electrocardiogram is not sufficient to meet the daily monitoring requirements. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a non-contact electrocardiogram monitoring method and system based on millimeter-wave sensing to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present application provides a non-contact electrocardiogram monitoring method based on millimeter-wave sensing, including:

[0007] Non-contact acquisition of the current original signal of the target to be measured from the chest area of the target to be measured in a millimeter-wave sensing manner;

[0008] Perform frequency-domain filtering on the original signal in a beamforming manner to extract the complex signal corresponding to the cardiac region of the target to be measured;

[0009] Input the complex signal into a preset electrocardiogram cross-modal mapping model so that the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the target to be measured.

[0010] In some embodiments of the present application, the non-contact acquisition of the current original signal of the target to be measured from the thoracic region of the target to be measured in a millimeter-wave sensing manner includes:

[0011] Control the millimeter-wave radar to transmit a linearly frequency-modulated signal with a linearly increasing frequency from its transmitting antenna to the thoracic region of the target to be measured based on the FMCW technology, and then receive the corresponding reflected signal from its receiving antenna;

[0012] Mix the transmitted linearly frequency-modulated signal and the reflected signal to generate the current digital intermediate-frequency signal of the target to be measured as the original signal.

[0013] In some embodiments of the present application, the millimeter-wave radar is configured with 2 transmitting antennas and 4 receiving antennas;

[0014] Correspondingly, the control of the millimeter-wave radar to transmit a linearly frequency-modulated signal with a linearly increasing frequency from its transmitting antenna to the thoracic region of the target to be measured based on the FMCW technology, and then receive the corresponding reflected signal from its receiving antenna, includes:

[0015] Control the millimeter-wave radar to alternately transmit a linearly frequency-modulated signal with a linearly increasing frequency from the 2 transmitting antennas to the thoracic region of the target to be measured in a time-division multiplexing manner, and then receive the corresponding reflected signal from its 4 receiving antennas.

[0016] In some embodiments of the present application, the frequency-domain filtering of the original signal in a beamforming manner to extract the complex signal corresponding to the cardiac region of the target to be measured includes:

[0017] Construct a multi-channel virtual antenna array according to the positional relationship between the transmitting antenna and the receiving antenna of the millimeter-wave radar;

[0018] Based on the virtual antenna array, perform frequency-domain filtering on the original signal in a beamforming manner to extract the complex signal corresponding to the cardiac region of the target to be measured.

[0019] In some embodiments of the present application, the frequency-domain filtering of the original signal in a beamforming manner based on the virtual antenna array to extract the complex signal corresponding to the cardiac region of the target to be measured includes:

[0020] Based on the virtual antenna array, each of the original signals is respectively mapped into a three-dimensional space by beamforming to extract the radar signal components corresponding to each of the original signals;

[0021] Frequency domain filtering is respectively performed on each of the radar signal components to locate and extract the corresponding cardiac activity information;

[0022] According to the cardiac activity information, in the three-dimensional coordinates corresponding to each of the radar signal components, a three-dimensional coordinate with the highest cardiac pulsation signal energy is selected, and the in-phase component and the quadrature component of the I / Q domain data corresponding to the three-dimensional coordinate are extracted, so as to determine the in-phase component and the quadrature component as the complex signal corresponding to the cardiac region of the target to be measured.

[0023] In some embodiments of the present application, the electrocardiogram cross-modal mapping model adopts a complex variational autoencoder network architecture;

[0024] The complex variational autoencoder network architecture includes:

[0025] An in-phase encoder, which is used to perform convolution operation, downsampling and fitting non-linear operation on the in-phase component of the input I / Q domain data to correspondingly output the millimeter-wave heartbeat feature corresponding to the in-phase component;

[0026] A quadrature encoder, which is used to perform convolution operation, downsampling and fitting non-linear operation on the quadrature component of the input I / Q domain data to correspondingly output the millimeter-wave heartbeat feature corresponding to the quadrature component;

[0027] A self-attention module, which is used to perform fusion processing on the millimeter-wave heartbeat feature corresponding to the in-phase component and the millimeter-wave heartbeat feature corresponding to the quadrature component to obtain the corresponding fused millimeter-wave heartbeat feature;

[0028] A decoder, which is used to reduce the dimension of the fused millimeter-wave heartbeat feature to output the electrocardiogram waveform data corresponding to the I / Q domain data.

[0029] In some embodiments of the present application, before inputting the complex signal into a preset electrocardiogram cross-modal mapping model, it further includes:

[0030] Obtain a plurality of historical original signals, which are non-contactively collected from the chest regions of different historical targets to be measured in advance by millimeter-wave sensing;

[0031] Based on beamforming, frequency domain filtering is performed on each of the historical original signals to extract the historical complex signals corresponding to the cardiac regions of different historical targets to be measured;

[0032] Using each of the historical complex signals as training data, training a preset complex variational autoencoder network, and supervising the training process of the complex variational autoencoder network using real historical electrocardiogram data collected synchronously with each of the historical original signals, so as to train the complex variational autoencoder network into an electrocardiogram cross-modal mapping model for cross-modal mapping of complex signals into electrocardiogram waveform data.

[0033] Another aspect of the present application provides a non-contact electrocardiogram monitoring system based on millimeter-wave sensing, including:

[0034] A data acquisition module for non-contact acquisition of the current original signal of the target to be measured from the chest region of the target to be measured in a millimeter-wave sensing manner;

[0035] A data processing module for frequency-domain filtering of the original signal based on beamforming to extract the complex signal corresponding to the cardiac region of the target to be measured;

[0036] A model mapping module for inputting the complex signal into a preset electrocardiogram cross-modal mapping model, so that the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the target to be measured.

[0037] Another aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the non-contact electrocardiogram monitoring method based on millimeter-wave sensing is implemented.

[0038] Another aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the non-contact electrocardiogram monitoring method based on millimeter-wave sensing is implemented.

[0039] The non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided by this application collects the current original signal of the target to be measured in a non-contact manner from the chest area of the target to be measured in a millimeter-wave sensing manner, can monitor the fine-grained cardiac activities of the human body without contact, and on the basis of being able to obtain reliable and effective electrocardiogram monitoring results of the subsequent target to be measured, and on the basis of being able to reconstruct the electrocardiogram waveform of the target to be measured when different subjects are sitting or lying supine, further improves the accuracy and effectiveness of the electrocardiogram monitoring results; by extracting the complex signal corresponding to the cardiac region, it is possible to avoid factors such as electromagnetic wave frequency offset and amplitude change that may be caused by ignoring cardiac electrical activities, and thus can effectively improve the comprehensiveness and reliability of the focus on cardiac activities, can effectively improve the extraction accuracy of cardiac activity information, and thus can provide a more accurate and comprehensive data basis for obtaining electrocardiogram monitoring results based on the complex signal, so as to further improve the accuracy and reliability of non-contact electrocardiogram monitoring; by inputting the complex signal into a preset electrocardiogram cross-modal mapping model so that the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the target to be measured, it is possible to effectively avoid losing key information of cardiac activities such as signal amplitude and frequency offset, can focus on the fine-grained activity information of the heart and reconstruct the electrocardiogram waveform, and thus can effectively improve the reliability of the non-contact electrocardiogram monitoring process and the accuracy of the electrocardiogram monitoring results.

[0040] Additional advantages, objects, and features of this application will be partly set forth in the description below, and will partly become apparent to those of ordinary skill in the art after study of the following, or may be learned from practice of this application. The objects and other advantages of this application may be realized and attained by the structure particularly pointed out in the specification and the drawings.

[0041] Those skilled in the art will understand that the objects and advantages that can be achieved by this application are not limited to the above specifically described, and the above and other objects that can be achieved by this application will be more clearly understood according to the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of this application, form a part of this application, and do not limit this application. The components in the drawings are not drawn to scale, but are only for showing the principle of this application. For the convenience of showing and describing some parts of this application, the corresponding parts in the drawings may be enlarged, that is, may become larger relative to other components in the exemplary device actually manufactured according to this application. In the drawings:

[0043] Figure 1 It is a first flow schematic diagram of the non-contact electrocardiogram monitoring method based on millimeter-wave sensing in an embodiment of this application.

[0044] Figure 2This is the second specific process schematic diagram of the non-contact electrocardiogram monitoring method based on millimeter-wave sensing in an embodiment of the present application.

[0045] Figure 3 This is the schematic diagram of the architecture of the electrocardiogram cross-modal mapping model in an embodiment of the present application.

[0046] Figure 4 This is the structural schematic diagram of the non-contact electrocardiogram monitoring system based on millimeter-wave sensing in another embodiment of the present application.

[0047] Figure 5 This is the schematic diagram of the overall logical process example of the non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided in the application example of the present application.

[0048] Figure 6 This is the schematic diagram of the process example of constructing a virtual antenna array and data processing process in the non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided in the application example of the present application. Detailed implementation manners

[0049] To make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the present application will be further described in detail below in conjunction with the implementation manners and the accompanying drawings. Herein, the illustrative implementation manners of the present application and their descriptions are used to explain the present application, but do not limit the present application.

[0050] Herein, it should also be noted that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the solution of the present application are shown in the drawings, while other details less related to the present application are omitted.

[0051] It should be emphasized that the term "including / containing" when used herein refers to the presence of features, elements, steps, or components, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0052] Herein, it should also be noted that if not specifically stated, the term "connection" in this document can not only refer to a direct connection, but also represent an indirect connection with an intermediate.

[0053] In the following, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0054] In one or more embodiments of the present application, ECG (Electrocardiogram) refers to an electrocardiogram, which is a graph recording the electrical activity changes generated during each cardiac cycle of the heart; mmWave (Millimeter Wave) refers to millimeter wave, which is an electromagnetic wave with a wavelength in the millimeter range; FMCW (Frequency Modulated Continuous Wave) refers to frequency modulated continuous wave, which is a radar modulation mode; Tx (Transmit) is used to refer to the transmit antenna of the radar; Rx (Receive) is used to refer to the receive antenna of the radar; FFT (Fast Fourier Transform) refers to fast Fourier transform; BF (Beamforming) refers to beamforming, which is used for the transmission or reception of electromagnetic wave directional focusing; MIMO (Multiple-in Multiple-out) refers to multiple antennas, a multiple-input multiple-output technology; DNN (Deep Neural Network) refers to deep neural network; Range FFT refers to range FFT, which is used for signal processing of FMCW radar to distinguish observed objects at different distances; Range Bin refers to range bin, and Range FFT separates observed objects at different distances into different range bins.

[0055] Electrocardiogram (ECG) is an important clinical method for monitoring the health status of the human heart and diagnosing cardiovascular diseases. By attaching lead electrodes to the skin of the subject, an electrocardiograph can record the electrical activity changes of the heart, including depolarization and repolarization of the atria and ventricles, etc. Electrocardiogram can be used for monitoring the vital signs of patients in various wards, and also for monitoring diseases such as arrhythmia and myocardial infarction in the clinic. In addition to its application in the clinic, daily electrocardiogram monitoring outside the clinic is of great significance for the screening, diagnosis and treatment of paroxysmal diseases. With the development of intelligent health technology, the development of electrocardiogram devices tends to be miniaturized and intelligent, and portable dynamic electrocardiogram recorders, single-lead electrocardiographs, etc. have emerged.

[0056] However, existing medical devices still cannot get rid of the dependence on attached electrode patches. During the electrocardiogram (ECG) monitoring process, electrodes need to be worn, which causes skin discomfort and inconvenience in use. This also makes it impossible for some special groups of people to wear electrode patches due to physical limitations, such as burn patients and newborns. In addition, the use of electrode patches also makes it impossible for the device to continuously and unobtrusively monitor the ECG for a long time, which makes it difficult to popularize ECG monitoring on a daily basis and impossible to obtain long-term ECG data. Non-contact ECG monitoring methods can solve these inconvenient problems, provide users with unobtrusive continuous ECG monitoring, record long-term cardiac health data, and thus achieve early screening and early warning prediction of diseases. As a non-contact sensing device, millimeter-wave radar has received wide attention in the field of health monitoring in recent years, and it has the potential ability to continuously and unobtrusively monitor the ECG.

[0057] The activities of the heart can be divided into electrical activities and mechanical activities. Among them, mechanical activities are caused by the conduction of electrical activities and myocardial excitation. Both of these activities can be monitored by the radar emitting electromagnetic waves. The electromagnetic waves emitted by the transmitting antenna will be reflected by the chest cavity back to the receiving antenna of the radar. The reflected signal carries information about the heart activities, which can be extracted through a series of algorithms and modeled as an ECG signal. Among them, the electrical activities of the heart will generate an electromagnetic field, which causes the frequency of the electromagnetic waves to shift during transmission. And the mechanical activities during the cardiac cycle cause minute displacement fluctuations in the chest cavity, which leads to regular changes in the phase of the electromagnetic waves.

[0058] In an ideal situation, traditional signal processing methods can extract the phase changes and frequency offsets caused by heart activities by focusing on specific distances and angles. However, the phase changes belong to the information of heart mechanical activities, and the frequency offsets belong to the information of magnetic field changes caused by heart activities. These information are different from the ECG signal information of the heart, and it is difficult to directly extract the corresponding ECG signal. In addition, during daily monitoring, the height and body shape of each subject are different, and the position of focusing on the heart will also change. And traditional signal processing methods do not have the ability to locate the heart. These factors make it difficult for traditional signal processing methods to accurately implement non-contact ECG monitoring.

[0059] The general process of existing non-contact ECG monitoring is as follows:

[0060] 1. Use a millimeter-wave radar to emit millimeter-wave signals to the object to be measured and receive the echo signals;

[0061] 2. Perform signal processing on the received echo signals, focus on the heart area of the echo through beamforming, and extract the heart mechanical activity data hidden in the echo signals;

[0062] 3. For the extracted heart mechanical activity data, construct an end-to-end algorithm to complete the cross-domain mapping from heart mechanical activities to heart electrical activities;

[0063] 4. Based on the algorithm that has learned the cross-domain mapping between cardiac mechanical activity and cardiac electrical activity, input the cardiac mechanical activity data extracted at the current moment, output the ECG measurement result at the current moment, and finally complete non-contact electrocardiogram monitoring.

[0064] However, existing technical solutions focus on the cardiac mechanical activity area according to fixed rule spectral features through beamforming. The focusing and extraction accuracy of this method for cardiac activity are insufficient. In addition, existing methods only extract the mechanical activity information of the heart in the radar reflection signal, while ignoring the possible electromagnetic wave frequency shift caused by cardiac electrical activity. These two factors result in insufficient consistency between the electrocardiogram monitoring waveform and the gold standard device during the implementation of existing technology, and the electrocardiogram accuracy is insufficient to meet the daily monitoring requirements.

[0065] Based on this, the embodiments of the present application provide a non-contact electrocardiogram monitoring method based on millimeter-wave sensing that can be implemented by a non-contact electrocardiogram monitoring system based on millimeter-wave sensing. See Figure 1 The non-contact electrocardiogram monitoring method based on millimeter-wave sensing specifically includes the following contents:

[0066] Step 100: Non-contact collect the current original signal of the measured target from the thoracic region of the measured target in a millimeter-wave sensing manner.

[0067] In one or more embodiments of the present application, the measured target may refer to a person currently undergoing electrocardiogram monitoring, or may also refer to other living bodies with electrocardiogram monitoring requirements, such as mammals such as cats and dogs, so that the non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided by the present application can be widely applicable.

[0068] In step 100, millimeter-wave sensing can be specifically implemented in a non-contact form by using a millimeter-wave radar. By using the millimeter-wave radar to detect the thoracic vibration of the measured target caused by the heartbeat, an original signal containing information such as the cardiac activity of the measured target can be obtained.

[0069] In one or more embodiments of the present application, the original signal contains all the information of the measured target sensed by millimeter waves, such as all information including respiration, heartbeat, and body movement.

[0070] Based on this, by non-contact collecting the current original signal of the measured target from the thoracic region of the measured target in a millimeter-wave sensing manner, the fine-grained cardiac activity of the human body can be monitored without contact, and it can provide a reliable and effective basis for obtaining the electrocardiogram monitoring result of the measured target in the future. Furthermore, on the basis that the system can reconstruct the electrocardiogram waveform of the measured target when different subjects are sitting or lying supine, the accuracy and effectiveness of the electrocardiogram monitoring result can be further improved.

[0071] In order to further improve the convenience and efficiency of collecting the original reflected signals of the thoracic region of the measured target, in a specific embodiment of the present application, an embedded millimeter-wave radar can also be used to monitor the fine-grained cardiac activities of the human body without contact.

[0072] Step 200: Perform frequency-domain filtering on the original reflected signals based on beamforming to extract the complex signals corresponding to the cardiac region of the measured target.

[0073] In step 200, performing frequency-domain filtering on the original reflected signals based on beamforming to extract the complex signals corresponding to the cardiac region of the measured target specifically means: performing spatial-domain filtering on the original reflected signals received by the radar through beamforming technology, and selecting the echo signals of the cardiac region to extract the complex signals corresponding to the cardiac region of the measured target.

[0074] It can be understood that the complex signals corresponding to the cardiac region refer to the complex I / Q signals containing cardiac mechanical activity information, and this complex I / Q signal can also be referred to as the complex I / Q domain signal or the I / Q domain complex signal.

[0075] Since the influence of cardiac pulsation on radar signals is not limited to the signal phase change caused by the minute undulation of the chest cavity, but also includes a series of influences such as frequency offset and amplitude change, etc., therefore, in step 200 of the present application, by extracting the complex signals corresponding to the cardiac region, it is possible to avoid ignoring factors such as electromagnetic wave frequency offset and amplitude change that may be caused by cardiac electrical activities, and thus can effectively improve the comprehensiveness and reliability of the focus on cardiac activities, can effectively improve the extraction accuracy of cardiac activity information, and further can provide a more accurate and comprehensive data basis for obtaining the electrocardiogram monitoring results based on this complex signal, so as to further improve the accuracy and reliability of non-contact electrocardiogram monitoring.

[0076] Step 300: Input the complex signals into a preset electrocardiogram cross-modal mapping model so that the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the measured target.

[0077] In one or more embodiments of the present application, the electrocardiogram monitoring result data refers to electrocardiogram waveform data for displaying the fine-grained activity information of the heart, and can specifically be output in the form of electrocardiogram data.

[0078] It can be understood that after the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the measured target, the electrocardiogram monitoring result data can be sent to the client device to enable the direct display or printing of the electrocardiogram monitoring result data, thereby further improving the usage experience of the measured target.

[0079] In step 300, the electrocardiogram cross-modal mapping model refers to a deep neural network DNN that is pre-trained based on historical complex signals and is used to cross-modally map complex signals into electrocardiogram waveform data. It can effectively avoid losing key information of cardiac activities such as signal amplitude and frequency deviation, can focus on fine-grained cardiac activity information and reconstruct the electrocardiogram waveform, and thus can effectively improve the reliability of the non-contact electrocardiogram monitoring process and the accuracy of electrocardiogram monitoring results.

[0080] From the above description, it can be seen that the non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided by the embodiments of the present application can avoid losing key information of cardiac activities such as electromagnetic wave frequency deviation and amplitude change caused by cardiac electrical activities, can effectively improve the comprehensiveness and reliability of focusing on cardiac activities, and can effectively improve the extraction accuracy of cardiac activity information, and thus can effectively improve the reliability and accuracy of non-contact electrocardiogram monitoring.

[0081] In order to further improve the reliability and effectiveness of collecting the original reflection signal of the chest area of the measured target, in a non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided by the embodiments of the present application, see Figure 2 , step 100 in the non-contact electrocardiogram monitoring method based on millimeter-wave sensing specifically includes the following contents:

[0082] Step 110: Control the millimeter-wave radar to transmit a linearly frequency-modulated signal with a linearly increasing frequency from its transmitting antenna to the chest area of the measured target, and then receive the corresponding reflection signal from its receiving antenna.

[0083] It can be understood that the reflection signal refers to the linearly frequency-modulated signal with a linearly increasing frequency that is reflected from the chest area of the measured target after transmitting the linearly frequency-modulated signal with a linearly increasing frequency to the chest area of the measured target.

[0084] Step 120: Mix the transmitted linearly frequency-modulated signal and the reflection signal to generate the current digital intermediate-frequency signal of the measured target as the original signal.

[0085] In order to further provide a reliable and effective data basis for the signal processing process of using complex signals to improve the focusing accuracy of cardiac activities, in a non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided by the embodiments of the present application, the millimeter-wave radar is configured with 2 transmitting antennas and 4 receiving antennas; correspondingly, step 110 in the non-contact electrocardiogram monitoring method based on millimeter-wave sensing also specifically includes the following contents:

[0086] Step 111: Control the millimeter-wave radar to alternately transmit linearly frequency-modulated signals with linearly increasing frequencies from the two transmitting antennas to the chest area of the target to be measured in a time-division multiplexing manner, and then receive the corresponding reflected signals from its four receiving antennas.

[0087] In order to further improve the effectiveness and reliability of extracting the complex signals corresponding to the heart area of the target to be measured, in a non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided in an embodiment of the present application, refer to Figure 2 , the step 200 in the non-contact electrocardiogram monitoring method based on millimeter-wave sensing specifically includes the following contents:

[0088] Step 210: Construct a multi-channel virtual antenna array according to the positional relationship between the transmitting antenna and the receiving antenna of the millimeter-wave radar.

[0089] In step 210, a virtual antenna array (i.e., an 8-channel virtual array) can be specifically constructed according to the antenna arrangement and positional relationship of the millimeter-wave radar for beamforming.

[0090] Step 220: Based on the virtual antenna array, perform frequency-domain filtering on the original signal by means of beamforming to extract the complex signals corresponding to the heart area of the target to be measured.

[0091] Specifically, through the above steps 110 to 220, the embodiment of the present application aims to sense the human heart activity by orienting the millimeter-wave antenna towards the chest position. Utilize the multi-antenna, multiple-input multiple-output technology (MIMO) of the millimeter-wave radar, including time-division multiplexing of the transmitting antenna and the virtual antenna array, to realize the acquisition of multi-channel radar signals. Combine the arrangement settings of the virtual antenna and the beamforming technology to model the space sensed by the radar.

[0092] In order to further improve the effectiveness and reliability of extracting the complex signals corresponding to the heart area of the target to be measured, in a non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided in an embodiment of the present application, the step 220 in the non-contact electrocardiogram monitoring method based on millimeter-wave sensing specifically includes the following contents:

[0093] Step 221: Based on the virtual antenna array, use the beamforming method to map each of the original signals into a three-dimensional space to extract the radar signal components corresponding to each of the original signals respectively.

[0094] In step 221, based on the virtual antenna array, use the beamforming method to map the information sensed by the radar into a three-dimensional space to extract each radar signal component.

[0095] Specifically, the multi-antenna arrangement of the millimeter-wave radar is specially designed, and the relative positions between the antennas are fixed. Beamforming can utilize the relative position information between the multi-antennas to map the information sensed by the radar into the three-dimensional space respectively, and the radar signal component Map(x, y, z) at that location can be extracted according to the three-dimensional coordinates (x, y, z):

[0096]

[0097] where i represents the antenna channel (2 transmitting antennas and 4 receiving antennas can provide 8 antenna channels), x i and y i represent the corresponding coordinates of the spatial position where the i-th channel is located, and Bin(z) represents the radar signal component in the range bin at the z distance.

[0098] Step 222: Perform frequency-domain filtering on each of the radar signal components to locate and extract the corresponding cardiac activity information.

[0099] In step 222, frequency-domain filtering is performed on each of the radar signal components that have undergone beamforming processing to locate and extract the cardiac activity information.

[0100] It can be understood that the cardiac activity information includes information such as the amplitude and frequency offset of the electrocardiogram signal, which is used for the extraction of the electrocardiogram in subsequent links.

[0101] Step 223: According to the cardiac activity information, in the three-dimensional coordinates corresponding to each of the radar signal components, select a three-dimensional coordinate with the highest cardiac pulsation signal energy, and extract the in-phase component and the quadrature component of the I / Q domain data corresponding to this three-dimensional coordinate, so as to determine the in-phase component and the quadrature component as the complex signal corresponding to the cardiac region of the measured target.

[0102] In step 223, first, the radar signal component Map(x, y, z) under each three-dimensional coordinate is converted into a phase signal;

[0103] Then, calculate the power spectral density (Power Spectral Density) of the phase signal under each three-dimensional coordinate, that is, the power spectral density of the signal phase of Map(x, y, z), denoted as PSD(x, y, z, f). PSD(x, y, z, f) represents the signal component at the (x, y, z) coordinate, and its power at the frequency f, and the unit of f is (times / minute). This frequency is the phase change frequency, that is, the frequency of the minute displacement of the chest cavity.

[0104] After that, calculate the sum F heart (x, y, z) of the power spectral density of the phase signal under each three-dimensional coordinate in the heart rate frequency band (40 - 200 times / minute), which is the cardiac pulsation signal energy:

[0105]

[0106] F heart (x, y, z) The higher it is, the higher the cardiac pulsation energy of the signal at the position coordinates, and the closer the position is to the heart. To determine the position coordinates where the cardiac pulsation is located, we take F heart (x, y, z) with the largest value of the coordinates (x 0 , y 0 , z 0 ), which is used for subsequent signal processing.

[0107]

[0108] Finally, extract the in-phase and quadrature components of the I / Q domain data:

[0109] After determining the specific coordinates (x 0 , y 0 , z 0 ), take the real part and the imaginary part of the complex signal at this coordinate (x 0 , y 0 , z 0 ) as the in-phase and quadrature components of the I / Q domain data respectively.

[0110] In order to further avoid losing key information such as signal amplitude and frequency offset of cardiac activities, in a non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided in an embodiment of the present application, the electrocardiogram cross-modal mapping model adopts a complex variational autoencoder network architecture;

[0111] See Figure 3 , the complex variational autoencoder network architecture includes:

[0112] An in-phase encoder, which is used to perform convolution operation, downsampling and fitting non-linear operation on the in-phase component of the input I / Q domain data to correspondingly output the millimeter-wave cardiac beat feature corresponding to the in-phase component;

[0113] A quadrature encoder, which is used to perform convolution operation, downsampling and fitting non-linear operation on the quadrature component of the input I / Q domain data to correspondingly output the millimeter-wave cardiac beat feature corresponding to the quadrature component;

[0114] A self-attention module (which can also be written as a self-attention mechanism), which is used to perform fusion processing on the millimeter-wave cardiac beat feature corresponding to the in-phase component and the millimeter-wave cardiac beat feature corresponding to the quadrature component to obtain the corresponding fused millimeter-wave cardiac beat feature;

[0115] A decoder, which is used to reduce the dimension of the fused millimeter-wave cardiac beat feature to output the electrocardiogram waveform data corresponding to the I / Q domain data.

[0116] In order to further improve the effectiveness and reliability of extracting the complex signals corresponding to the cardiac regions of the measured targets, in a non-contact electrocardiogram monitoring method based on millimeter-wave sensing provided in an embodiment of the present application, refer to Figure 2 , before step 300, before step 100, and at other times in the non-contact electrocardiogram monitoring method based on millimeter-wave sensing, the following content may specifically be included:

[0117] Step 010: Obtain a plurality of historical original signals, where the historical original signals are non-contact collected from the thoracic regions of different historical measured targets in advance in a millimeter-wave sensing manner.

[0118] Step 020: Perform frequency-domain filtering on each of the historical original signals based on beamforming to extract the historical complex signals corresponding to the cardiac regions of different historical measured targets.

[0119] Step 030: Use each of the historical complex signals as training data to train a preset complex variational autoencoder network, and use the real historical electrocardiogram data collected synchronously with each of the historical original signals to supervise the training process of the complex variational autoencoder network, so as to train the complex variational autoencoder network into an electrocardiogram cross-modal mapping model for cross-modal mapping of complex signals into electrocardiogram waveform data.

[0120] Specifically, the electrocardiogram cross-modal mapping model can be trained using the I / Q domain data obtained by signal processing in the foregoing embodiments, and the electrocardiogram data collected synchronously by a medical device when receiving radar signals is used as the gold standard for supervision. During the training process, the weighted sum of binary cross-entropy loss and KL divergence (Kullback-Leibler divergence) is used as the loss function to optimize and improve the ability of the model to extract heartbeat features from I / Q domain data and reconstruct electrocardiogram waveforms.

[0121] From a software perspective, the present application also provides a non-contact electrocardiogram monitoring system based on millimeter-wave sensing for executing all or part of the non-contact electrocardiogram monitoring method based on millimeter-wave sensing, refer to Figure 4 , the non-contact electrocardiogram monitoring system based on millimeter-wave sensing specifically includes the following content:

[0122] A data acquisition module 10, configured to non-contact collect the current original signal of the measured target from the thoracic region of the measured target in a millimeter-wave sensing manner.

[0123] A data processing module 20, configured to perform frequency-domain filtering on the original signal based on beamforming to extract the complex signal corresponding to the cardiac region of the measured target.

[0124] The model mapping module 30 is configured to input the complex signal into a preset electrocardiogram cross-modal mapping model, so that the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the target to be measured.

[0125] The embodiments of the non-contact electrocardiogram monitoring system based on millimeter-wave sensing provided in this application can specifically be used to execute the processing procedures of the embodiments of the non-contact electrocardiogram monitoring method based on millimeter-wave sensing in the above embodiments. Its functions will not be elaborated here, and reference can be made to the detailed description of the embodiments of the non-contact electrocardiogram monitoring method based on millimeter-wave sensing above.

[0126] The part of the non-contact electrocardiogram monitoring based on millimeter-wave sensing by the non-contact electrocardiogram monitoring system based on millimeter-wave sensing can be executed in the client device. Specifically, it can be selected according to the processing capabilities of the client device and the limitations of the user usage scenario, etc. This application does not make any limitations in this regard. If all operations are completed in the client device, the client device may further include a processor and a millimeter-wave radar for the specific processing of non-contact electrocardiogram monitoring based on millimeter-wave sensing.

[0127] The above-mentioned client device may have a communication module (i.e., a communication unit), and can be communicatively connected to a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, it may also include a server of an intermediate platform, such as a server of a third-party server platform having a communication link with the task scheduling center server. The server may include a single computer device, or may include a server cluster composed of multiple servers, or a server structure of a distributed device.

[0128] Any suitable network protocol can be used for communication between the above server and the client device side, including network protocols not yet developed on the filing date of this application. The network protocol may, for example, include TCP / IP protocol, UDP / IP protocol, HTTP protocol, HTTPS protocol, etc. Of course, the network protocol may, for example, also include RPC protocol (Remote Procedure Call Protocol) and REST protocol (Representational State Transfer) used on top of the above protocols.

[0129] As can be seen from the above description, the non-contact electrocardiogram monitoring system based on millimeter-wave sensing provided by the embodiments of the present application can avoid key information of cardiac activities such as electromagnetic wave frequency offset and amplitude change that may be caused by the loss of cardiac electrical activities, can effectively improve the comprehensiveness and reliability of the focus on cardiac activities, and can effectively improve the extraction accuracy of cardiac activity information, thereby effectively improving the reliability and accuracy of non-contact electrocardiogram monitoring.

[0130] To further illustrate the present solution, the present application also provides a specific application example of a non-contact electrocardiogram monitoring method and system based on millimeter-wave sensing. Taking the target human body as an example to be measured, it specifically relates to a method and system for capturing ECG records without contacting the human body, which works by analyzing the reflection of millimeter-wave radar signals on the human body, and designs a neural network model for radar signal processing. The influence of cardiac pulsation on radar signals is not limited to the signal phase change caused by the slight undulation of the chest cavity, but also includes a series of influences such as frequency offset and amplitude change. Therefore, the present application designs a complex signal encoder for extracting the characteristic information of cardiac pulsation. In addition, since there are many noises in the detection range of the radar, the present application designs the neural network structure of an electrocardiogram monitoring decoder to focus on the fine-grained activity information of the heart and reconstruct the electrocardiogram waveform.

[0131] See Figure 5 and Figure 6 , the non-contact electrocardiogram monitoring method based on millimeter-wave sensing implemented by using the non-contact electrocardiogram monitoring system based on millimeter-wave sensing is specifically described as follows:

[0132] (1) Data acquisition

[0133] S1: The millimeter-wave radar receives the original signal: The millimeter-wave radar (subsequently abbreviated as: radar) is used to detect the chest cavity vibration of the target human body caused by the heartbeat, and the original signal containing information such as the cardiac activity of the target human body is obtained.

[0134] Specifically, step S1 in the data recording link may specifically include the following content:

[0135] S11: The millimeter-wave radar uses the FMCW technology to transmit a linearly frequency-modulated signal with a linearly increasing frequency from its transmitting antenna to the target human body, and then the millimeter-wave radar uses the FMCW technology to collect the reflected signal containing information such as the cardiac activity of the target human body from its receiving antenna.

[0136] Among them, in order to provide an effective data basis for the subsequent data processing process of using complex signals to improve the focusing accuracy of cardiac activities, the millimeter-wave radar can be configured with multiple transmitting antennas for alternately transmitting linearly frequency-modulated signals with a linearly increasing frequency in a time-division multiplexing manner.

[0137] S12: Mix the linear frequency modulation signal (i.e., FMCW signal) with linearly increasing frequency transmitted and received by the millimeter-wave radar to generate a digital intermediate frequency (IF) signal, which serves as the original signal containing information such as the cardiac activity of the target human body for subsequent processing.

[0138] It can be understood that the original signal contains all the information of the target human body sensed by the radar, including all information such as respiration, heartbeat, and body movement.

[0139] In an example, the millimeter-wave radar can be configured with 2 transmitting antennas and 4 receiving antennas, and the 2 transmitting antennas alternately transmit signals in a time-division multiplexing manner. That is, the 2 transmitting antennas of the radar alternately transmit FMCW waves (i.e., linear frequency modulation signals with linearly increasing frequency) in a time-division multiplexing mode, and the receiving antennas are deployed in a 4-channel array to output a digital intermediate frequency signal as the original signal.

[0140] (II) Data Processing

[0141] S2: Focus on the echo of the cardiac region and extract the complex signal: Perform spatial filtering on the original reflected signal received by the radar through beamforming technology, select the echo signal of the cardiac region, and extract the complex I / Q domain signal containing information on cardiac mechanical activity.

[0142] Specifically, step S2 in the data processing link may specifically include the following content:

[0143] S21: Construct an 8-channel virtual array based on the positional relationship between the antennas, and perform frequency domain filtering through beamforming to obtain the I / Q signal of the cardiac region. The specific description is as follows:

[0144] (1) Construct a virtual antenna array (i.e., 8-channel virtual array) based on the antenna arrangement and positional relationship of the radar for beamforming.

[0145] (2) Based on the virtual antenna array, use beamforming to map the information sensed by the radar into three-dimensional space to extract each radar signal component.

[0146] Specifically, the multi-antenna arrangement of the millimeter-wave radar is specially designed, and the relative positions between the antennas are fixed. Beamforming can utilize the relative position information between the multi-antennas to map the information sensed by the radar into three-dimensional space respectively, and the radar signal component Map(x, y, z) at that location can be extracted according to the three-dimensional coordinates (x, y, z):

[0147]

[0148] Among them, i represents the antenna channel (2 transmitting antennas and 4 receiving antennas can provide 8 antenna channels), x i and y iThe corresponding coordinates representing the spatial position of the i-th channel, and Bin(z) represents the radar signal component in the range bin at a distance of z.

[0149] (3) Perform frequency-domain filtering on each radar signal component after beamforming processing to locate and extract cardiac activity information.

[0150] It can be understood that the cardiac activity information includes information such as the amplitude and frequency offset of the electrocardiogram signal, which is used for the extraction of electrocardiograms in subsequent steps.

[0151] (4) According to the situation of locating cardiac activity information in the previous step, determine the beam signal of a specific coordinate and extract its I / Q components. Specifically as follows:

[0152] First, convert the radar signal component Map(x, y, z) at each three-dimensional coordinate into a phase signal;

[0153] Then calculate the power spectral density (Power Spectral Density) of the phase signal at each three-dimensional coordinate, that is, the power spectral density of the signal phase of Map(x, y, z), denoted as PSD(x, y, z, f). PSD(x, y, z, f) represents the signal component at the (x, y, z) coordinate, and its power at a frequency of f, and the unit of f is (times / minute). This frequency is the phase change frequency, that is, the frequency of the minute displacement of the chest cavity.

[0154] After that, calculate the sum F of the power spectral density of the phase signal at each three-dimensional coordinate in the heart rate frequency band (40 - 200 times / minute) heart (x, y, z), which is the cardiac pulsation signal energy:

[0155]

[0156] F heart (x, y, z) is higher, indicating that the cardiac pulsation energy in the signal at this position coordinate is higher and the distance from the heart is closer. To determine the position coordinate where the cardiac pulsation is located, we take the coordinate (x heart (x, y, z) with the largest value for subsequent signal processing. 0 , y 0 , z 0 ) for subsequent signal processing.

[0157]

[0158] S22: Extract the in-phase and quadrature components of the I / Q domain data. The specific description is as follows:

[0159] Determine the specific coordinate (x 0 , y 0 , z 0) After that, take the real and imaginary parts of the complex signals at the coordinates (x 0 , y 0 , z 0 ) as the in-phase and quadrature components of the I / Q domain data respectively.

[0160] S23: Respectively serve as the inputs of the I / Q encoder of the complex variational autoencoder network.

[0161] Specifically, use the in-phase and quadrature components obtained in S22 as the I / Q domain complex signals comprehensively representing cardiac activities for the model input of the next neural network stage, so as to use the complex variational autoencoder model module based on deep learning for electrocardiogram extraction and monitoring.

[0162] (III) Complex Variational Autoencoder Network

[0163] S3: Use the complex information (i.e., the I / Q domain complex signal obtained in S23) as the input, and map the radar echo back to the electrocardiogram signal through the electrocardiogram cross-modal mapping model;

[0164] S4: Output the electrocardiogram result.

[0165] That is: Import the I / Q domain complex signal into the trained electrocardiogram cross-modal mapping model for electrocardiogram cross-modal mapping, and output the reconstructed electrocardiogram waveform.

[0166] Among them, the electrocardiogram cross-modal mapping model adopts the complex variational autoencoder network architecture, which is used to separate the pure electrocardiogram signal from the complex signal containing cardiac activities and reconstruct the ECG waveform. The overall model of the complex variational autoencoder presents a "bottleneck" structure, including three main parts: an encoder, an attention mechanism, and a decoder.

[0167] Among them, the complex variational autoencoder network architecture is trained using the historical I / Q domain data obtained in the same way as the signal processing process in the foregoing embodiments, and the electrocardiogram data synchronously collected by the medical device when the radar receives the signal is used as the gold standard for supervision. During the training process, the weighted sum of binary cross-entropy loss and KL divergence (Kullback-Leibler divergence) is used as the loss function to optimize and improve the model's ability to extract heartbeat features from I / Q domain data and reconstruct the electrocardiogram waveform, and finally form an electrocardiogram cross-modal mapping model for outputting electrocardiogram signals.

[0168] Specifically, the function description of the electrocardiogram cross-modal mapping model adopting the complex variational autoencoder network architecture is as follows:

[0169] (1) The encoder encodes the high-dimensional input into low-dimensional latent variables, enabling the neural network to learn the most informative features. The in-phase component and the quadrature component of the I / Q signal are respectively fed into the in-phase encoder and the quadrature encoder. The two parallel encoders have the same structure and both contain a one-dimensional convolutional operation layer for processing time-series signals. The convolutional layer will perform downsampling through the stride design. The encoder also includes a non-linear mapping and a batch normalization layer to fit non-linear operations and accelerate the model convergence process. The final output of the encoder is the millimeter-wave heartbeat features of the I / Q two channels.

[0170] (2) The self-attention mechanism is used to focus on the electrocardiogram-related information after the concatenation of the I / Q two-channel features, realizing the fusion processing of complex signals. The decoder restores the latent variables of the hidden layer to the initial dimension and reconstructs the expected output to achieve the purpose of electrocardiogram waveform reconstruction.

[0171] (3) The decoder is the inverted structure of the encoder, including a one-dimensional transposed convolutional operation layer, a batch normalization layer, and non-linear mapping, etc. The one-dimensional transposed convolution will achieve the purpose of upsampling through the stride design. The output of the decoder is the reconstructed electrocardiogram waveform.

[0172] The complex variational autoencoder model designed in this application has the following specific characteristics:

[0173] 1. Two parallel encoders with the same structure are used to process the in-phase component and the quadrature component of the I / Q signal respectively, achieving the purpose of complex signal processing. It no longer relies solely on phase to extract cardiac activities, avoiding the loss of electrocardiogram-related information such as amplitude and frequency offset. The parallel encoders can also accelerate the convergence speed and improve the training efficiency;

[0174] 2. The encoded features (i.e., low-dimensional latent vectors) of the encoder are extracted in the form of a probability distribution, that is, a variational encoder. This design can avoid overfitting of the network and improve the generalization ability of the network;

[0175] 3. For the two-channel features of the in-phase component and the quadrature component, the self-attention mechanism is used for fusion, realizing deep fusion and making the network structure more suitable for complex signals;

[0176] 4. The decoder is implemented using a transposed convolution network structure to reconstruct the complex signal features into an electrocardiogram waveform.

[0177] The non-contact electrocardiogram monitoring method and system based on millimeter-wave sensing proposed in the application example of this application can use an embedded millimeter-wave radar to monitor the fine-grained cardiac activities of the human body without contact and output an electrocardiogram to achieve non-contact electrocardiogram monitoring. The system can reconstruct the ECG waveform when different subjects are sitting or lying supine. The waveform has strong consistency with the gold standard device and can be used in scenarios such as health detection and medical diagnosis.

[0178] Compared with the prior art, the beneficial effects of the application examples of the present application are as follows:

[0179] 1. To achieve the perception of human heart activities, the millimeter-wave antenna is oriented towards the chest position. By using the multi-antenna, multiple-input multiple-output technology (MIMO) of millimeter-wave radar, including time-division multiplexing of transmitting antennas and virtual antenna arrays, the acquisition of multi-channel radar signals is realized. Combining with the arrangement of virtual antennas and beamforming technology, the space sensed by the radar is modeled.

[0180] 2. The radar senses a large spatial range with a lot of noise. To focus on the heart movement in the sensing space, based on the frequency-domain information of the spatial position signals, the coordinates of the heart activities in the spatial model are located, and the complex signals at that position are extracted.

[0181] 3. Using a deep neural network DNN to extract electrocardiogram waveforms, a complex variational autoencoder network architecture is proposed. Benefiting from the parallel encoder structure and attention mechanism therein, it can achieve parallel processing and deep fusion of the in-phase component and quadrature component in the complex signal, so as to achieve the purpose of extracting heart activity characteristics. This method avoids losing key information of heart activities such as signal amplitude and frequency offset. In addition, the decoder structure in the neural network architecture can reconstruct the electrocardiogram waveform according to the above-extracted characteristics and use it as the final output.

[0182] The embodiment of the present application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the non-contact electrocardiogram monitoring method based on millimeter-wave sensing mentioned in the above embodiment. The processor and the memory can be connected through a bus or other means. Taking the connection through the bus as an example. The receiver can be connected to the processor and the memory in a wired or wireless manner.

[0183] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or a combination of the above types of chips.

[0184] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the non-contact electrocardiogram monitoring method based on millimeter-wave sensing in the embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory, that is, to implement the non-contact electrocardiogram monitoring method based on millimeter-wave sensing in the above method embodiments.

[0185] The memory may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created by the processor, etc. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely provided with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0186] The one or more modules are stored in the memory and, when executed by the processor, execute the non-contact electrocardiogram monitoring method based on millimeter-wave sensing in the embodiments.

[0187] In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, the memory, the receiver, and the transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to transmit and receive signals.

[0188] As an implementation manner, the functions of the receiver and the transmitter in the present application can be considered to be implemented through a transceiver circuit or a dedicated transceiver chip, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit, or a general-purpose chip.

[0189] As another implementation manner, it can be considered to use a general-purpose computer to implement the server provided in the embodiments of the present application. That is, the program codes for implementing the functions of the processor, the receiver, and the transmitter are stored in the memory, and the general-purpose processor implements the functions of the processor, the receiver, and the transmitter by executing the codes in the memory.

[0190] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the foregoing non-contact electrocardiogram monitoring method based on millimeter-wave sensing are implemented. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0191] Those of ordinary skill in the art should understand that the various exemplary components, systems, and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Specifically, whether to implement in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments used to execute the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave on a transmission medium or a communication link.

[0192] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, the detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0193] In the present application, the features described and / or illustrated for one embodiment can be used in the same way or in a similar way in one or more other embodiments, and / or combined with the features of other embodiments or replace the features of other embodiments.

[0194] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A non-contact electrocardiogram monitoring method based on millimeter-wave sensing, characterized in that, it includes: Non-contact acquisition of the current original signal of the measured target from the chest area of the measured target in a millimeter-wave sensing manner; Frequency-domain filtering of the original signal based on beamforming to extract the complex signal corresponding to the heart area of the measured target; Inputting the complex signal into a preset electrocardiogram cross-modal mapping model so that the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the measured target; The non-contact acquisition of the current original signal of the measured target from the chest area of the measured target in a millimeter-wave sensing manner includes: Controlling the millimeter-wave radar to transmit a linearly frequency-modulated signal with linearly increasing frequency from its transmitting antenna to the chest area of the measured target based on FMCW technology, and then receiving the corresponding reflected signal from its receiving antenna; Mixing the transmitted linearly frequency-modulated signal and the reflected signal to generate the current digital intermediate-frequency signal of the measured target as the original signal; The millimeter-wave radar is configured with 2 transmitting antennas and 4 receiving antennas; Correspondingly, the controlling the millimeter-wave radar to transmit a linearly frequency-modulated signal with linearly increasing frequency from its transmitting antenna to the chest area of the measured target based on FMCW technology, and then receiving the corresponding reflected signal from its receiving antenna includes: Controlling the millimeter-wave radar to alternately transmit linearly frequency-modulated signals with linearly increasing frequency from the 2 transmitting antennas to the chest area of the measured target in a time-division multiplexing manner, and then receiving the corresponding reflected signals from its 4 receiving antennas; The frequency-domain filtering of the original signal based on beamforming to extract the complex signal corresponding to the heart area of the measured target includes: Constructing a multi-channel virtual antenna array according to the positional relationship between the transmitting antenna and the receiving antenna of the millimeter-wave radar; Based on the virtual antenna array, frequency-domain filtering the original signal by beamforming to extract the complex signal corresponding to the heart area of the measured target; The frequency-domain filtering of the original signal based on the virtual antenna array by beamforming to extract the complex signal corresponding to the heart area of the measured target includes: Based on the virtual antenna array, using beamforming to map each original signal into three-dimensional space respectively to extract the radar signal components corresponding to each original signal respectively; Performing frequency-domain filtering on each of the radar signal components respectively to locate and extract the corresponding heart activity information; According to the heart activity information, in the three-dimensional coordinates corresponding to each of the radar signal components, selecting a three-dimensional coordinate with the highest heart beat signal energy, and extracting the in-phase component and the quadrature component of the I / Q domain data corresponding to the three-dimensional coordinate, so as to determine the in-phase component and the quadrature component as the complex signal corresponding to the heart area of the measured target.

2. The non-contact electrocardiogram monitoring method based on millimeter-wave sensing according to claim 1, characterized in that, the electrocardiogram cross-modal mapping model adopts a complex variational autoencoder network architecture; The complex variational autoencoder network architecture includes: In-phase encoder, which is used to perform convolution operation, downsampling and fitting non-linear operation on the in-phase component of the input I / Q domain data to correspondingly output the millimeter-wave heartbeat feature corresponding to the in-phase component; Quadrature encoder, which is used to perform convolution operation, downsampling and fitting non-linear operation on the quadrature component of the input I / Q domain data to correspondingly output the millimeter-wave heartbeat feature corresponding to the quadrature component; Self-attention module, which is used to perform fusion processing on the millimeter-wave heartbeat feature corresponding to the in-phase component and the millimeter-wave heartbeat feature corresponding to the quadrature component to obtain the corresponding fused millimeter-wave heartbeat feature; Decoder, which is used to perform dimensionality reduction on the fused millimeter-wave heartbeat feature to output the electrocardiogram waveform data corresponding to the I / Q domain data.

3. The non-contact electrocardiogram monitoring method based on millimeter-wave sensing according to claim 1 or 2, characterized in that, before inputting the complex signal into the preset electrocardiogram cross-modal mapping model, it further includes: acquiring a plurality of historical original signals, where the historical original signals are non-contactively collected from the chest regions of different historical measured targets in a millimeter-wave sensing manner; performing frequency-domain filtering on each of the historical original signals based on beamforming to extract the historical complex signals corresponding to the heart regions of different historical measured targets; using each of the historical complex signals as training data to train a preset complex variational autoencoder network, and supervising the training process of the complex variational autoencoder network with the real historical electrocardiogram data synchronously collected with each of the historical original signals, so as to train the complex variational autoencoder network into an electrocardiogram cross-modal mapping model for cross-modal mapping of complex signals into electrocardiogram waveform data.

4. A non-contact electrocardiogram monitoring system based on millimeter-wave sensing, characterized in that, it includes: Data acquisition module, which is used to non-contactively collect the current original signal of the measured target from the chest region of the measured target in a millimeter-wave sensing manner; Data processing module, which is used to perform frequency-domain filtering on the original signal based on beamforming to extract the complex signal corresponding to the heart region of the measured target; Model mapping module, which is used to input the complex signal into a preset electrocardiogram cross-modal mapping model, so that the electrocardiogram cross-modal mapping model outputs the electrocardiogram monitoring result data of the measured target; wherein, the non-contactively collecting the current original signal of the measured target from the chest region of the measured target in a millimeter-wave sensing manner includes: controlling a millimeter-wave radar to transmit a linearly frequency-modulated signal with a linearly increasing frequency from its transmitting antenna to the chest region of the measured target, and then receiving the corresponding reflected signal from its receiving antenna; mixing the transmitted linearly frequency-modulated signal and the reflected signal to generate the current digital intermediate-frequency signal of the measured target as the original signal; the millimeter-wave radar is configured with 2 transmitting antennas and 4 receiving antennas; Correspondingly, the control millimeter-wave radar transmits a linearly frequency-modulated signal with a linearly increasing frequency from its transmitting antenna to the chest area of the target to be measured based on the FMCW technology, and then receives the corresponding reflected signal from its receiving antenna, including: The control millimeter-wave radar alternately transmits linearly frequency-modulated signals with linearly increasing frequencies from the 2 transmitting antennas to the chest area of the target to be measured in a time-division multiplexing manner based on the FMCW technology, and then receives the corresponding reflected signals from its 4 receiving antennas; The method of performing frequency-domain filtering on the original signal based on beamforming to extract the complex signal corresponding to the heart area of the target to be measured includes: Construct a multi-channel virtual antenna array according to the positional relationship between the transmitting antenna and the receiving antenna of the millimeter-wave radar; Based on the virtual antenna array, perform frequency-domain filtering on the original signal through beamforming to extract the complex signal corresponding to the heart area of the target to be measured; The method of performing frequency-domain filtering on the original signal based on the virtual antenna array through beamforming to extract the complex signal corresponding to the heart area of the target to be measured includes: Based on the virtual antenna array, use beamforming to map each of the original signals into three-dimensional space respectively to extract the radar signal components corresponding to each of the original signals; Perform frequency-domain filtering on each of the radar signal components respectively to locate and extract the corresponding heart activity information; According to the heart activity information, in the three-dimensional coordinates corresponding to each of the radar signal components, select a three-dimensional coordinate with the highest heart beat signal energy, and extract the in-phase component and the quadrature component of the I / Q domain data corresponding to the three-dimensional coordinate, so as to determine the in-phase component and the quadrature component as the complex signal corresponding to the heart area of the target to be measured.

5. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the non-contact electrocardiogram monitoring method based on millimeter-wave sensing according to any one of claims 1 to 3.

6. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, it implements the non-contact electrocardiogram monitoring method based on millimeter-wave sensing according to any one of claims 1 to 3.

Images