Cyclic generative adversarial network reconstruction method and device based on non-contact electrocardiosignals
Through the cyclic generation adversarial network reconstruction method of contactless ECG signals, the Cycle-GAN network is used to perform cross-modal conversion, which solves the problem of time-frequency joint characteristics and timing mismatch in contactless ECG signals, and realizes accurate reconstruction from radar signals to ECG signals.
Patent Information
- Application Number
- CN202510658878.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-21
- Publication Date
- 2025-08-12
AI Technical Summary
The prior art fails to fully utilize the time-frequency joint characteristics in contactless electrocardiogram signal monitoring, and the radar-ECG cross-modal conversion method fails to effectively solve the timing mismatch between electrophysiological activities and mechanical micro-movement, making it difficult to accurately reconstruct the ECG signal.
The cyclic generation adversarial network reconstruction method based on contactless ECG signals is adopted. By extracting and encoding the contactless ECG radar signals, the Gram angular difference field image of the ECG radar radar is generated, and the image reconstruction model is constructed. The Cycle-GAN network is used for cross-modal conversion, and the ECG signal is finally reconstructed.
Accurate reconstruction from radar signal to ECG signal is achieved, the problem of failure to fully utilize the time-frequency joint characteristics and timing mismatch in the prior art is solved, and a new idea of contactless vital sign perception is provided.
Smart Images
Figure CN120458591A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of electrocardiogram (ECG) signal monitoring, and more specifically to a method and apparatus for reconstructing a cyclic generative adversarial network based on non-contact ECG signals. Background Art
[0002] Vital sign monitoring, as a crucial component of human health management, has garnered increasing attention in the healthcare sector. In recent years, non-contact radar sensing has demonstrated significant potential in electrocardiogram (ECG) signal monitoring. As a non-contact sensing technology, radar not only acquires patient status information without physical contact, but also protects privacy and is unaffected by ambient brightness, enabling monitoring under varying lighting conditions. This makes radar technology promising for non-contact vital sign monitoring. It addresses the challenges of traditional contact electrode monitoring, such as poor comfort, limited portability, and limited applicability. While traditional methods based on signal transformation and correlation analysis can capture periodic heartbeats, they are unable to reconstruct ECG (electrocardiogram) signals from radar echoes. Currently, common approaches for radar signal reconstruction include time-frequency conversion, end-to-end deep learning models, and physiological parameter profile estimation.
[0003] However, in practice, it is found that when the above method is used to reconstruct radar signals, the following technical problems often occur: First, existing research directly processes one-dimensional signals without using high-dimensional signal analysis methods, and fails to fully utilize the joint time-frequency features; Second, existing radar-ECG cross-modal conversion methods generally do not fully consider the timing mismatch between electrophysiological activity and mechanical micromotion. Summary of the Invention
[0004] The content of this application is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this application is not intended to identify key features or essential features of the technical solution for which protection is sought, nor is it intended to limit the scope of the technical solution for which protection is sought.
[0005] Some embodiments of the present application propose a method, apparatus, computer device, and computer-readable storage medium for reconstructing a cyclic generative adversarial network based on non-contact electrocardiogram signals to solve one or more of the technical problems mentioned in the above background technology section.
[0006] In a first aspect, some embodiments of the present application provide a recurrent generative adversarial network reconstruction method based on non-contact ECG signals, the method comprising: performing component extraction processing on a pre-collected non-contact ECG radar signal to obtain an ECG radar characteristic signal; performing encoding processing on the ECG radar characteristic signal to obtain an ECG radar Gram angle difference field image; performing pre-processing on a pre-collected sample radar signal set and a label ECG signal set, respectively, to obtain a sample radar Gram angle difference field image set and a label ECG Gram angle difference field image set; constructing an image reconstruction model based on the sample radar Gram angle difference field image set and the label ECG Gram angle difference field image set; inputting the ECG radar Gram angle difference field image into the image reconstruction model to obtain a reconstructed ECG image; and performing conversion processing on the reconstructed ECG image to obtain a reconstructed ECG signal.
[0007] In a second aspect, some embodiments of the present disclosure provide a recurrent generative adversarial network reconstruction device based on non-contact electrocardiographic signals, the device comprising: a component extraction unit, configured to perform component extraction processing on a pre-collected non-contact electrocardiographic radar signal to obtain an electrocardiographic radar characteristic signal; an encoding unit, configured to perform encoding processing on the above-mentioned electrocardiographic radar characteristic signal to obtain an electrocardiographic radar Gram angle difference field image; a preprocessing unit, configured to preprocess the pre-collected sample radar signal set and the label electrocardiographic signal set respectively to obtain a sample radar Gram angle difference field image set and a label electrocardiographic Gram angle difference field image set; a construction unit, configured to construct an image reconstruction model based on the above-mentioned sample radar Gram angle difference field image set and the above-mentioned label electrocardiographic Gram angle difference field image set; an input unit, configured to input the above-mentioned electrocardiographic radar Gram angle difference field image into the above-mentioned image reconstruction model to obtain a reconstructed electrocardiographic image; and a conversion unit, configured to perform conversion processing on the above-mentioned reconstructed electrocardiographic image to obtain a reconstructed electrocardiographic signal.
[0008] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the method described in any implementation of the first aspect.
[0009] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the method described in any implementation of the first aspect is implemented.
[0010] The aforementioned embodiments of the present application have the following beneficial effects: The recurrent generative adversarial network reconstruction method based on non-contact ECG signals, as described in some embodiments of the present application, can effectively reconstruct ECG signals, providing a new approach for contactless vital sign sensing. Specifically, the difficulty in effectively reconstructing ECG signals lies in the following reasons: first, existing research directly processes one-dimensional signals without using high-dimensional signal analysis methods, thus failing to fully utilize joint time-frequency features; and second, existing radar-ECG cross-modal conversion methods generally fail to fully consider the timing mismatch between electrophysiological activity and mechanical micromotion. Based on this, the recurrent generative adversarial network reconstruction method based on non-contact ECG signals, as described in some embodiments of the present application, first performs component extraction processing on pre-collected non-contact ECG radar signals to obtain ECG radar feature signals. Second, the ECG radar feature signals are encoded to obtain ECG radar Gram angle difference field images. Next, the pre-collected sample radar signal set and the label ECG signal set are pre-processed to obtain sample radar Gram angle difference field images and label ECG Gram angle difference field images. Then, an image reconstruction model is constructed based on the sample radar Gram angle difference field image set and the labeled ECG Gram angle difference field image set. Subsequently, the ECG radar Gram angle difference field image is input into the image reconstruction model to obtain a reconstructed ECG image. Finally, the reconstructed ECG image is transformed to obtain a reconstructed ECG signal. Thus, the present application proposes several methods for reconstructing non-contact ECG signals using a cyclic generative adversarial network. First, a time-frequency standardization method for reconstructing non-contact ECG signals using a cyclic generative adversarial network is proposed. This method utilizes the joint time-frequency characteristics of radar signals through dimensionality reduction, redundancy removal, and two-dimensional graph conversion. Second, a radar-ECG cross-modality conversion method is proposed, utilizing the cycle consistency constraints of a Cycle-GAN (Cycle Generative Adversarial Network) network to address the timing mismatch between electrophysiological activity and mechanical micromotion. Finally, a Cycle-GAN ECG reconstruction network that integrates non-contact ECG signals is proposed, achieving nonlinear mapping from radar signals to ECG signals. Therefore, this paper proposes a contactless ECG signal reconstruction method based on millimeter-wave radar and Cycle-GAN. This method combines a generative adversarial network with time-frequency coding to achieve effective reconstruction of the radar signal into the ECG signal. This technique removes redundant information from the radar signal and converts the one-dimensional time-series radar signal into a two-dimensional image using the Gram angle difference field. The Cycle-GAN model then learns the mapping between the radar signal and the corresponding two-dimensional image of the ECG signal, thereby achieving accurate ECG signal reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The above and other features, advantages, and aspects of the various embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the elements and components are not necessarily drawn to scale.
[0012] Figure 1 is a flowchart of some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals according to the present application; Figure 2 is a schematic diagram of radar signals and electrocardiogram signals according to some embodiments of the method for reconstructing a cyclic generative adversarial network based on non-contact electrocardiogram signals of the present application; Figure 3 is a schematic diagram of the results of component extraction of radar signals measured in a resting state according to some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact electrocardiogram signals of the present application; Figure 4 2. It is a schematic diagram of the results of component extraction of radar signals measured in an apnea state according to some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals of the present application; Figure 5 This is a schematic diagram of the results of component extraction of radar signals measured in a strong closed call state according to some embodiments of the non-contact electrocardiogram signal-based cyclic generative adversarial network reconstruction method of the present application; Figure 6 Schematic diagram of ECG radar Gram angle difference field images corresponding to radar signals in three states according to some embodiments of the recurrent generative adversarial network reconstruction method based on non-contact ECG signals of the present application; Figure 7 Schematic diagram of reconstructed ECG images corresponding to radar signals in three states according to some embodiments of the recurrent generative adversarial network reconstruction method based on non-contact ECG signals of the present application; Figure 8 is a flowchart of some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals according to the present application; Figure 9 is a schematic diagram of an application scenario of some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact electrocardiogram signals according to the present application; Figure 10 1 is a schematic structural diagram of some embodiments of a cyclic generative adversarial network reconstruction device based on non-contact electrocardiographic signals according to the present disclosure; Figure 11 It is a schematic diagram of the structure of a computer device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION
[0013] The following will describe embodiments of the present application in more detail with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.
[0014] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of this application can be combined with each other.
[0015] It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0016] It should be noted that the modifications of "one" and "multiple" mentioned in this application are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".
[0017] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0018] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.
[0019] Figure 1 The flowchart 100 of some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals according to the present application is shown. The cyclic generative adversarial network reconstruction method based on non-contact ECG signals includes the following steps: Step 101 : performing component extraction processing on the pre-collected non-contact ECG radar signal to obtain an ECG radar characteristic signal.
[0020] In some embodiments, the execution entity of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals performs component extraction processing on pre-collected non-contact ECG radar signals to obtain ECG radar characteristic signals. The non-contact ECG radar signals may be radar signals collected by millimeter-wave radars. The non-contact ECG radar signals may represent the heart rate characteristics of a target user in a target physiological state. The target user may be a user performing ECG signal collection. The target physiological state may be, but is not limited to, a resting state, an apnea state, or a strong closed-call state. The non-contact ECG radar signals include a channel signal set. The channel signals in the channel signal set may represent signals emitted by a transmitting channel of the millimeter-wave radar.
[0021] As an example, the millimeter wave radar can be a 24 GHz (gigahertz) continuous wave radar based on six-port interferometry technology. The non-contact electrocardiogram radar signal can be referenced. Figure 2 Schematic diagram of radar signal and ECG signal according to some embodiments of the non-contact ECG signal Cycle-GAN (Cycle Generative Adversarial Network) reconstruction method integrating PCA (principal components analysis) and Gram angle difference field of the present application. Figure 2 As shown, Figure 2 (a) in the figure can represent the ECG radar signal in a resting state. Figure 2 (b) in the figure can represent the ECG signal in a resting state. Figure 2 (c) in the figure can represent the ECG radar signal in the apnea state. Figure 2 (d) in the figure can represent the ECG signal in the apnea state. Figure 2 The (e) in the figure can represent the ECG radar signal in the strong closed call state. Figure 2 (f) in the figure can represent the ECG signal in the strong closed breathing state.
[0022] In some optional implementations of some embodiments, the execution subject performs component extraction processing on the pre-collected non-contact ECG radar signal to obtain the ECG radar characteristic signal, which may include the following steps: The first step is to perform normalization processing on the non-contact ECG radar signal to obtain a standard ECG radar signal. The normalization processing can be performed using a preset normalization algorithm. The standard ECG radar signal can include a standard channel signal set. Each standard channel signal in the standard channel signal set can represent a signal received by a channel of a millimeter-wave radar receiver.
[0023] As an example, the preset normalization algorithm may be a Z-score (standard deviation) normalization algorithm.
[0024] The second step is to determine the electrocardiogram radar covariance matrix corresponding to the electrocardiogram radar standard signal. The electrocardiogram radar covariance matrix corresponding to the electrocardiogram radar standard signal can be determined by the following formula: .
[0025] in, represents the ECG radar covariance matrix. Indicates the number of standard channel signals in the above standard channel signal set. Indicates the standard ECG radar signal.
[0026] The third step is to determine the set of electrocardiogram radar eigenvalues and the set of electrocardiogram radar eigenvectors corresponding to the electrocardiogram radar covariance matrix. The electrocardiogram radar eigenvalues in the set of electrocardiogram radar eigenvalues and the electrocardiogram radar eigenvectors in the set of electrocardiogram radar eigenvectors correspond one-to-one. In practice, the set of electrocardiogram radar eigenvalues and the set of electrocardiogram radar eigenvectors corresponding to the electrocardiogram radar covariance matrix can be determined by eigenvalue solution.
[0027] In the fourth step, the individual electrocardiogram radar eigenvalues in the electrocardiogram radar eigenvalue set are sorted to obtain a sequence of electrocardiogram radar eigenvalues. The sequence of electrocardiogram radar eigenvalues can be obtained by sorting the individual electrocardiogram radar eigenvalues in the set from largest to smallest. The electrocardiogram radar eigenvalues in the set can represent a principal component feature in the electrocardiogram radar standard signal.
[0028] In a fifth step, the cardiology radar feature vectors in the set of cardiology radar feature vectors corresponding to the number of cardiology radar feature values preceding the target in the cardiology radar feature value sequence are combined to obtain a cardiology radar projection matrix. The step of combining the cardiology radar feature vectors in the set of cardiology radar feature vectors corresponding to the number of cardiology radar feature values preceding the target in the cardiology radar feature value sequence to obtain the cardiology radar projection matrix may be performed by determining the cardiology radar feature vectors in the set of cardiology radar feature vectors corresponding to the number of cardiology radar feature values preceding the target in the cardiology radar feature value sequence as vectors in the cardiology radar projection matrix to obtain the cardiology radar projection matrix.
[0029] As an example, the target number may be, but is not limited to, at least one of the following: 3, 5, or 7.
[0030] In a sixth step, based on the electrocardiogram radar projection matrix, dimensionality reduction processing is performed on the electrocardiogram radar standard signal to obtain an electrocardiogram radar characteristic signal. The dimensionality reduction processing on the electrocardiogram radar projection matrix to obtain the electrocardiogram radar characteristic signal may be performed by multiplying the electrocardiogram radar standard signal by the electrocardiogram radar projection matrix to determine the electrocardiogram radar characteristic signal.
[0031] As an example, the above component extraction process is performed on the pre-collected non-contact ECG radar signal to obtain the intermediate result of the ECG radar characteristic signal, which can be referred to Figure 3-5 Specifically, Figure 3 The following is a schematic diagram showing the results of component extraction of radar signals measured in a resting state according to some embodiments of the recurrent generative adversarial network reconstruction method based on non-contact ECG signals of the present application. Figure 3 As shown, Figure 3 (a) in the figure represents the standard ECG radar signal in a resting state. Figure 3 (b) in the figure shows the principal component features represented by the number of target ECG radar eigenvalues in the resting state. Here, the number of targets is 5. Figure 3 (c) in FIG. 5 shows a principal component scatter plot of a non-contact electrocardiographic radar signal in a resting state, where PC1 represents the first principal component axis, PC2 represents the second principal component axis, and PC3 represents the third principal component axis. Figure 3 (d) in the figure represents the variance explained by each principal component in the resting state. Here, single refers to a single principal component, and accumulation refers to the accumulated amount. Figure 3 (e) in the figure shows the spatial distribution of principal components in the resting state. Figure 3 (f) in the figure represents the ECG radar characteristic signal in the resting state. Here, original signal represents the non-contact ECG radar signal, and reconstructed signal represents the reconstructed ECG signal.
[0032] Figure 4 The following is a schematic diagram showing the results of component extraction of radar signals measured in an apnea state according to some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals of the present application. Figure 4 As shown, Figure 4 (a) in the figure shows the standard ECG radar signal in the apnea state. Figure 4 (b) shows the principal component features represented by the number of target ECG radar eigenvalues before the ECG radar eigenvalue sequence in the apnea state. Here, the number of targets is 5. Figure 4(c) in FIG. 1 shows a principal component scatter plot of a non-contact electrocardiographic radar signal in an apnea state, where PC1 represents the first principal component axis, PC2 represents the second principal component axis, and PC3 represents the third principal component axis. Figure 4 (d) in the figure shows the variance explanation rate of each principal component in the apnea state. Figure 4 (e) in the figure shows the spatial distribution of principal components in the apnea state. Figure 4 (f) in the figure represents the characteristic ECG radar signal in the apnea state.
[0033] Figure 5 The following is a schematic diagram showing the results of component extraction of radar signals measured in a strong closed call state according to some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals of the present application. Figure 5 As shown, Figure 5 (a) in the figure represents the standard ECG radar signal in the strong closed call state. Figure 5 (b) shows the principal component features of the ECG radar eigenvalue sequence before the target number in the strong closed call state. Here, the number of targets is 5. Figure 5 (c) in FIG. 1 shows a principal component scatter plot of the non-contact ECG radar signal in the strong closed call state, where PC1 represents the first principal component axis, PC2 represents the second principal component axis, and PC3 represents the third principal component axis. Figure 5 (d) in the figure represents the variance explanation rate of each principal component under the strong closed breath state. Figure 5 (e) in the figure represents the spatial distribution of principal components under strong closed breathing state. Figure 5 (f) in the figure represents the characteristic signal of the ECG radar in the strong closed call state.
[0034] Step 102: Encode the ECG radar characteristic signal to obtain an ECG radar Gram angle difference field image.
[0035] In some embodiments, the execution entity may perform encoding processing on the electrocardiographic radar characteristic signal to obtain an electrocardiographic radar Gram angle difference field image.
[0036] In some optional implementations of some embodiments, the execution subject encodes the electrocardiographic radar characteristic signal to obtain an electrocardiographic radar Gram angle difference field image, which may include the following steps: The first step is to normalize the electrocardiogram radar characteristic signal to obtain an electrocardiogram radar normalized signal. The normalization process can be performed on the electrocardiogram radar characteristic signal using a preset normalization algorithm to obtain the electrocardiogram radar normalized signal.
[0037] As an example, the above-mentioned preset normalization algorithm may be a maximum-minimum normalization algorithm.
[0038] The second step is to perform polar coordinate conversion processing on the normalized ECG radar signal to obtain a normalized ECG radar coordinate set. The polar coordinate conversion processing can be performed on the normalized ECG radar signal using a preset polar coordinate conversion algorithm to obtain the normalized ECG radar coordinate set.
[0039] As an example, the aforementioned preset polar coordinate conversion algorithm may be a Gram angular field algorithm.
[0040] The third step is to generate the cardio radar Gram angle difference field matrix based on the above-mentioned cardio radar normalized coordinate set. The cardio radar Gram angle difference field matrix can be generated based on the above-mentioned cardio radar normalized coordinate set using a preset generation formula.
[0041] As an example, the above-mentioned preset generation formula may be a GADF (Gramian Angular Difference Field) formula.
[0042] The fourth step is to transform the ECG radar Gram angle difference field matrix to obtain the ECG radar Gram angle difference field image. The transformation of the ECG radar Gram angle difference field matrix to obtain the ECG radar Gram angle difference field image may include: first, normalizing each element of the ECG radar Gram angle difference field matrix using the preset normalization algorithm to obtain a normalized ECG radar Gram angle difference field matrix. Then, the values of each element of the normalized ECG radar Gram angle difference field matrix are determined as the grayscale values of each pixel in the ECG radar Gram angle difference field image to obtain the ECG radar Gram angle difference field image.
[0043] As an example, the above ECG radar Gram angle difference field image can be referred to Figure 6 Schematic diagram of ECG radar Gram angle difference field images corresponding to radar signals in three states according to some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals of the present application. Figure 6 As shown, Figure 6 (a) in the figure shows the ECG radar Gram angle difference field image corresponding to the radar signal in the resting state. Figure 6 (b) in the figure shows the ECG radar Gram angle difference field image corresponding to the radar signal in the apnea state. Figure 6 (c) in the figure shows the ECG radar Gram angle difference field image corresponding to the radar signal in the strong closed call state.
[0044] Step 103 , pre-processing the pre-collected sample radar signal set and label ECG signal set respectively to obtain a sample radar Gram angle difference field image set and a label ECG Gram angle difference field image set.
[0045] In some embodiments, the execution entity may pre-process the pre-collected sample radar signal set and the label ECG signal set, respectively, to obtain a sample radar Gram angle difference field image set and a label ECG Gram angle difference field image set. The sample radar signals in the sample radar signal set correspond one-to-one to the label ECG signals in the label ECG signal set. The sample radar signals in the sample radar signal set may be radar signals of sample users collected by millimeter-wave radar. The label ECG signals in the label ECG signal set may be ECG signals of sample users collected by an electrocardiograph. The sample user may be the user used to collect the sample radar signals and the label ECG signals.
[0046] In some optional implementations of some embodiments, the execution entity preprocesses the pre-collected sample radar signal set and the label ECG signal set to obtain the sample radar Gram angle difference field image set and the label ECG Gram angle difference field image set, which may include the following steps: The first step is to perform component extraction processing on each sample radar signal in the sample radar signal set to generate a sample radar signature signal, thereby obtaining a sample radar signature signal set. The specific implementation of generating the sample radar signature signal set and the resulting technical effects can be found in step 101 of the above embodiment and will not be further described here.
[0047] The second step is to encode each sample radar characteristic signal in the sample radar characteristic signal set to generate a sample radar Gram angle difference field image, thereby obtaining a sample radar Gram angle difference field image set. The specific implementation method for generating the sample radar Gram angle difference field image set and the resulting technical effects can be found in step 102 of the above embodiment and will not be further described here.
[0048] In the third step, component extraction is performed on each label ECG signal in the label ECG signal set to generate a label ECG feature signal, thereby obtaining a label ECG feature signal set. The specific implementation method for generating the label ECG feature signal set and the resulting technical effects can be found in step 101 of the above embodiment and will not be further described here.
[0049] In the fourth step, each label ECG feature signal in the label ECG feature signal set is encoded to generate a label ECG Gram angle difference field image, thereby obtaining a label ECG Gram angle difference field image set. The specific implementation method for generating the label ECG Gram angle difference field image set and the resulting technical effects can be found in step 102 of the above embodiment and will not be further described here.
[0050] Step 104 : constructing an image reconstruction model based on the sample radar Gram angle difference field image set and the label ECG Gram angle difference field image set.
[0051] In some embodiments, the execution entity may construct an image reconstruction model based on the sample radar Gram angle difference field image set and the labeled ECG Gram angle difference field image set, wherein the sample radar Gram angle difference field images in the sample radar Gram angle difference field image set correspond one-to-one to the labeled ECG Gram angle difference field image set in the labeled ECG Gram angle difference field image set.
[0052] In some optional implementations of some embodiments, the execution entity constructs an image reconstruction model based on the sample radar Gram angle difference field image set and the labeled ECG Gram angle difference field image set, which may include the following steps: In the first step, a target sample radar Gram angle difference field image is selected from the above sample radar Gram angle difference field image set, and the following training sub-steps are performed: In a first sub-step, the target sample radar Gram angle difference field image is input into an initial image reconstruction model to obtain an initial radar reconstructed image. A sample radar Gram angle difference field image can be randomly selected from the set of sample radar Gram angle difference field images to serve as the target sample radar Gram angle difference field image. The initial image reconstruction model can be an untrained neural network model that takes the target sample radar Gram angle difference field image as input and outputs the initial radar reconstructed image.
[0053] As an example, the initial image reconstruction model may be a Cycle-GAN (Cycle-Consistent Adversarial Networks) model.
[0054] The second sub-step is to determine the reconstruction loss value of the labeled ECG Gram angle difference field image and the initial radar reconstructed image corresponding to the initial radar reconstructed image in the labeled ECG Gram angle difference field image set based on a preset loss function.
[0055] As an example, the above-mentioned preset loss function can be but is not limited to one of the following: a cycle consistency loss function, a cross entropy loss function, a least square function or a classification cross entropy loss function.
[0056] In a third sub-step, in response to determining that the reconstruction loss value is less than the target threshold, the initial image reconstruction model is determined as the image reconstruction model.
[0057] As an example, the target threshold mentioned above may be 0.01.
[0058] Optionally, the execution entity may further adjust relevant parameters in the initial image reconstruction model in response to determining that the reconstruction loss value is greater than or equal to a target threshold, determine the adjusted initial image reconstruction model as the initial image reconstruction model, and select a target sample radar Gram angle difference field image from each unselected sample radar Gram angle difference field image in the sample radar Gram angle difference field image set for re-executing the training step. The relevant parameters in the initial image reconstruction model may be adjusted using a preset adjustment algorithm.
[0059] As an example, the preset adjustment algorithm may be, but is not limited to, one of the following: a back propagation algorithm or a stochastic gradient algorithm.
[0060] Step 105 : Input the ECG radar Gram angle difference field image into an image reconstruction model to obtain a reconstructed ECG image.
[0061] In some embodiments, the execution entity may input the ECG radar Gram angle difference field image into the image reconstruction model to obtain a reconstructed ECG image. The image reconstruction model may be a trained neural network model that takes the ECG radar Gram angle difference field image as input and outputs the reconstructed ECG image.
[0062] As an example, the above reconstructed ECG image can be referred to Figure 7 Schematic diagram of reconstructed ECG images corresponding to radar signals in three states according to some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact ECG signals of the present application. Figure 7 As shown, Figure 7 (a) in the figure shows the reconstructed electrocardiogram corresponding to the radar signal in a resting state. Figure 7 (b) in the figure shows the reconstructed ECG image corresponding to the radar signal in the apnea state. Figure 7 (c) in the figure shows the reconstructed ECG image corresponding to the radar signal in the strong closed call state.
[0063] Therefore, the image reconstruction model can be used to convert the radar signal's Gram angle difference field image into an ECG signal's Gram angle difference field image. This image reconstruction model preserves richer details and automatically learns the mapping relationship through adversarial training and a cycle consistency loss, resulting in strong image-to-image generalizability. The cycle consistency loss in the image reconstruction model ensures that the transformation from radar Gram angle difference field image to ECG and back again reconstructs the original input, preventing the generator from losing key information and ensuring that the generated ECG signal retains the physiological characteristics of the radar signal. The image reconstruction model can not only generate ECG Gram angle difference field images from radar Gram angle difference field images, but also generate radar signal Gram angle difference field images from ECG signal Gram angle difference field images. This bidirectional mapping capability enhances the model's robustness and can be used for data augmentation or cross-domain analysis.
[0064] Step 106: convert the reconstructed ECG image to obtain a reconstructed ECG signal.
[0065] In some embodiments, the execution subject may perform conversion processing on the reconstructed ECG image to obtain a reconstructed ECG signal, wherein the conversion processing may be performed on the reconstructed ECG image to obtain a reconstructed ECG signal using a preset conversion algorithm.
[0066] As an example, the preset conversion algorithm may be a Gram's angle field algorithm.
[0067] The aforementioned embodiments of the present application have the following beneficial effects: The recurrent generative adversarial network reconstruction method based on non-contact ECG signals, as described in some embodiments of the present application, can effectively reconstruct ECG signals, providing a new approach for contactless vital sign sensing. Specifically, the difficulty in effectively reconstructing ECG signals lies in the following reasons: first, existing research directly processes one-dimensional signals without using high-dimensional signal analysis methods, thus failing to fully utilize joint time-frequency features; and second, existing radar-ECG cross-modal conversion methods generally fail to fully consider the timing mismatch between electrophysiological activity and mechanical micromotion. Based on this, the recurrent generative adversarial network reconstruction method based on non-contact ECG signals, as described in some embodiments of the present application, first performs component extraction processing on pre-collected non-contact ECG radar signals to obtain ECG radar feature signals. Second, the ECG radar feature signals are encoded to obtain ECG radar Gram angle difference field images. Next, the pre-collected sample radar signal set and the label ECG signal set are pre-processed to obtain sample radar Gram angle difference field images and label ECG Gram angle difference field images. Then, an image reconstruction model is constructed based on the sample radar Gram angle difference field image set and the labeled ECG Gram angle difference field image set. Subsequently, the ECG radar Gram angle difference field image is input into the image reconstruction model to obtain a reconstructed ECG image. Finally, the reconstructed ECG image is transformed to obtain a reconstructed ECG signal. Thus, the present application proposes several methods for reconstructing non-contact ECG signals using a cyclic generative adversarial network. First, a time-frequency standardization method for reconstructing non-contact ECG signals using a cyclic generative adversarial network is proposed. This method utilizes the joint time-frequency characteristics of radar signals through dimensionality reduction, redundancy removal, and two-dimensional graph conversion. Second, a radar-ECG cross-modality conversion method is proposed, utilizing the cycle consistency constraints of a Cycle-GAN (Cycle Generative Adversarial Network) network to address the timing mismatch between electrophysiological activity and mechanical micromotion. Finally, a Cycle-GAN ECG reconstruction network that integrates non-contact ECG signals is proposed, achieving nonlinear mapping from radar signals to ECG signals. Therefore, this paper proposes a contactless ECG signal reconstruction method based on millimeter-wave radar and Cycle-GAN. This method combines a generative adversarial network with time-frequency coding to achieve effective reconstruction of the radar signal into the ECG signal. This technique removes redundant information from the radar signal and converts the one-dimensional time-series radar signal into a two-dimensional image using the Gram angle difference field. The Cycle-GAN model then learns the mapping between the radar signal and the corresponding two-dimensional image of the ECG signal, thereby achieving accurate ECG signal reconstruction.
[0068] Figure 8 This is a flowchart of some embodiments of the cyclic generative adversarial network reconstruction method based on non-contact electrocardiogram signals according to the present application.
[0069] like Figure 8 As shown in the figure, the radar (millimeter-wave radar) first transmits a radar signal via the transmitter (Tx). The radar then receives a feedback signal via the receiver (Rx). The radar then sends the feedback signal to the DL model (Deep Learning Model). Simultaneously, the ECG device (electrocardiogram) sends the actual ECG signal (for training) to the DL model for training. The DL model then reconstructs the radar signal into an ECG reconstruction image. Finally, the ECG reconstruction is converted into a reconstructed ECG signal.
[0070] Figure 9 This is a schematic diagram of an application scenario of a cyclic generative adversarial network reconstruction method based on non-contact electrocardiogram signals in some embodiments of the present disclosure.
[0071] exist Figure 9 In an application scenario, a computing device can first perform component extraction processing on a pre-collected non-contact ECG radar signal to obtain an ECG radar characteristic signal. The pre-collected non-contact ECG radar signal can be a CW radar signal (Continuous Wave radar signal). The ECG radar characteristic signal can be a PCA signal. Secondly, the ECG radar characteristic signal can be encoded to obtain an ECG radar Gram angle difference field image. The ECG radar Gram angle difference field image can be a GACF image. Then, the pre-collected sample radar signal set and the labeled ECG signal set can be pre-processed to obtain a sample radar Gram angle difference field image set and a labeled ECG Gram angle difference field image set. Next, an image reconstruction model can be constructed based on the sample radar Gram angle difference field image set and the labeled ECG Gram angle difference field image set. The image reconstruction model can be a Cycle-GAN model. Subsequently, the ECG radar Gram angle difference field image can be input into the image reconstruction model to obtain a reconstructed ECG image. Finally, the reconstructed ECG image may be converted to obtain a reconstructed ECG signal, where the reconstructed ECG signal may be a Reconstructed ECG (reconstructed ECG signal).
[0072] Further references Figure 10 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a cyclic generative adversarial network reconstruction device based on non-contact ECG signals. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the cyclic generation adversarial network reconstruction device based on non-contact electrocardiogram signals can be specifically applied to various electronic devices.
[0073] like Figure 10 As shown, in some embodiments, a cyclic generative adversarial network reconstruction device 1000 based on non-contact electrocardiogram signals includes: a component extraction unit 1001, an encoding unit 1002, a preprocessing unit 1003, a construction unit 1004, an input unit 1005 and a conversion unit 1006. Among them, the component extraction unit 1001 is configured to perform component extraction processing on the pre-collected non-contact ECG radar signal to obtain the ECG radar characteristic signal; the encoding unit 1002 is configured to perform encoding processing on the above-mentioned ECG radar characteristic signal to obtain the ECG radar Gram angle difference field image; the preprocessing unit 1003 is configured to preprocess the pre-collected sample radar signal set and the label ECG signal set respectively to obtain the sample radar Gram angle difference field image set and the label ECG Gram angle difference field image set; the construction unit 1004 is configured to construct an image reconstruction model based on the above-mentioned sample radar Gram angle difference field image set and the above-mentioned label ECG Gram angle difference field image set; the input unit 1005 is configured to input the above-mentioned ECG radar Gram angle difference field image into the above-mentioned image reconstruction model to obtain a reconstructed ECG image; and the conversion unit 1006 is configured to perform conversion processing on the above-mentioned reconstructed ECG image to obtain a reconstructed ECG signal.
[0074] It can be understood that the units described in the non-contact ECG signal-based cyclic generation adversarial network reconstruction device 1000 are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the apparatus 1000 for reconstructing the cyclic generation adversarial network based on the non-contact ECG signal and the units contained therein, and will not be described in detail here.
[0075] This application also provides a computer device 1100. Figure 11 As shown, computer device 1100 includes a bus 1101, a processor 1102, a memory 1103, and a communication interface 1104. Processor 1102, memory 1103, and communication interface 1104 communicate with each other via bus 1101. Computer device 1100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computer device 1100.
[0076] The bus 1101 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 The bus 1101 may include a path for transmitting information between various components of the computer device 1100 (eg, the memory 1103, the processor 1102, and the communication interface 1104).
[0077] The processor 1102 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0078] The memory 1103 may include a volatile memory, such as a random access memory (RAM). The memory 1103 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0079] The memory 1103 stores executable program code, and the processor 1102 executes the executable program code to implement the functions of the aforementioned component extraction unit, encoding unit, preprocessing unit, construction unit, input unit, and conversion unit, thereby implementing the aforementioned recurrent generative adversarial network reconstruction method based on non-contact ECG signals. In other words, the memory 1103 stores instructions for executing the aforementioned recurrent generative adversarial network reconstruction method based on non-contact ECG signals.
[0080] The communication interface 1104 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computer device 1100 and other devices or a communication network.
[0081] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the above-mentioned cyclic generative adversarial network reconstruction method based on non-contact electrocardiogram signals.
[0082] Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, hard disk, or magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the aforementioned recurrent generative adversarial network reconstruction method based on non-contact ECG signals.
[0083] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0084] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.
Claims
1. A recurrent generative adversarial network reconstruction method based on non-contact electrocardiogram signals, comprising: Performing component extraction processing on the pre-collected non-contact ECG radar signal to obtain ECG radar characteristic signal; performing encoding processing on the electrocardiographic radar characteristic signal to obtain an electrocardiographic radar Gram angle difference field image; Preprocessing the pre-collected sample radar signal set and the label electrocardiogram signal set respectively to obtain a sample radar Gram angle difference field image set and a label electrocardiogram Gram angle difference field image set; constructing an image reconstruction model based on the sample radar Gram angle difference field image set and the label electrocardiogram Gram angle difference field image set; Inputting the ECG radar Gram angle difference field image into the image reconstruction model to obtain a reconstructed ECG image; The reconstructed electrocardiogram image is converted to obtain a reconstructed electrocardiogram signal.
2. The recurrent generative adversarial network reconstruction method based on non-contact ECG signals according to claim 1, wherein: The component extraction process of the pre-collected non-contact ECG radar signal to obtain the ECG radar characteristic signal includes: performing standardization processing on the non-contact ECG radar signal to obtain an ECG radar standard signal; Determining the electrocardiographic radar covariance matrix corresponding to the electrocardiographic radar standard signal; Determining an electrocardiogram radar eigenvalue set and an electrocardiogram radar eigenvector set corresponding to the electrocardiogram radar covariance matrix, wherein the electrocardiogram radar eigenvalues in the electrocardiogram radar eigenvalue set and the electrocardiogram radar eigenvectors in the electrocardiogram radar eigenvector set have a one-to-one correspondence; Sorting each electrocardiogram radar eigenvalue in the electrocardiogram radar eigenvalue set to obtain an electrocardiogram radar eigenvalue sequence; Combining the electrocardiogram radar feature vectors in the electrocardiogram radar feature vector set and corresponding to the electrocardiogram radar feature values of the number of preceding targets in the electrocardiogram radar feature value sequence to obtain an electrocardiogram radar projection matrix; Based on the electrocardiogram radar projection matrix, the electrocardiogram radar standard signal is subjected to dimensionality reduction processing to obtain an electrocardiogram radar characteristic signal.
3. The recurrent generative adversarial network reconstruction method based on non-contact ECG signals according to claim 1, wherein: The encoding process of the electrocardiographic radar characteristic signal to obtain the electrocardiographic radar Gram angle difference field image includes: performing normalization processing on the electrocardiogram radar characteristic signal to obtain an electrocardiogram radar normalized signal; Performing polar coordinate conversion processing on the electrocardiographic radar normalized signal to obtain an electrocardiographic radar normalized coordinate set; generating an electrocardiographic radar Gram angle difference field matrix based on the electrocardiographic radar normalized coordinate set; The electrocardiographic radar Gram angle difference field matrix is converted to obtain the electrocardiographic radar Gram angle difference field image.
4. The recurrent generative adversarial network reconstruction method based on non-contact ECG signals according to claim 1, wherein: The pre-processing of the pre-collected sample radar signal set and the label electrocardiogram signal set to obtain the sample radar Gram angle difference field image set and the label electrocardiogram Gram angle difference field image set includes: performing component extraction processing on each sample radar signal in the sample radar signal set to generate a sample radar feature signal, thereby obtaining a sample radar feature signal set; performing encoding processing on each sample radar characteristic signal in the sample radar characteristic signal set to generate a sample radar Gram angle difference field image, thereby obtaining a sample radar Gram angle difference field image set; performing component extraction processing on each label ECG signal in the label ECG signal set to generate a label ECG feature signal, thereby obtaining a label ECG feature signal set; Each label ECG feature signal in the label ECG feature signal set is coded to generate a label ECG Gram angle difference field image, thereby obtaining a label ECG Gram angle difference field image set.
5. The recurrent generative adversarial network reconstruction method based on non-contact ECG signals according to claim 1, wherein: The sample radar Gram angle difference field images in the sample radar Gram angle difference field image set correspond one-to-one to the label electrocardiogram Gram angle difference field images in the label electrocardiogram Gram angle difference field image set; and constructing an image reconstruction model based on the sample radar Gram angle difference field image set and the label electrocardiogram Gram angle difference field image set, including [A1] Select a target sample radar Gram angle difference field image from the sample radar Gram angle difference field image set, and perform the following training steps: Inputting the target sample radar Gram angle difference field image into the initial image reconstruction model to obtain the initial radar reconstructed image; Based on a preset loss function, determining the reconstruction loss values of the labeled ECG Gram angle difference field image and the initial radar reconstructed image corresponding to the initial radar reconstructed image in the labeled ECG Gram angle difference field image set; In response to determining that the reconstruction loss value is less than the target threshold, the initial image reconstruction model is determined as the image reconstruction model.
6. The recurrent generative adversarial network reconstruction method based on non-contact ECG signals according to claim 5, wherein: The method further comprises: In response to determining that the reconstruction loss value is greater than or equal to the target threshold, relevant parameters in the initial image reconstruction model are adjusted, the adjusted initial image reconstruction model is determined as the initial image reconstruction model, and a target sample radar Gram angle difference field image is selected from each unselected sample radar Gram angle difference field image in the sample radar Gram angle difference field image set for performing the training step again.
7. A recurrent generative adversarial network reconstruction device based on non-contact ECG signals, characterized in that: include: a component extraction unit configured to perform component extraction processing on the pre-collected non-contact electrocardiographic radar signal to obtain an electrocardiographic radar characteristic signal; an encoding unit configured to perform encoding processing on the electrocardiographic radar characteristic signal to obtain an electrocardiographic radar Gram angle difference field image; a preprocessing unit configured to preprocess the pre-collected sample radar signal set and the label electrocardiogram signal set respectively to obtain a sample radar Gram angle difference field image set and a label electrocardiogram Gram angle difference field image set; a construction unit configured to construct an image reconstruction model based on the sample radar Gram angle difference field image set and the label electrocardiogram Gram angle difference field image set; an input unit configured to input the ECG radar Gram angle difference field image into the image reconstruction model to obtain a reconstructed ECG image; The conversion unit is configured to perform conversion processing on the reconstructed electrocardiogram image to obtain a reconstructed electrocardiogram signal.
8. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.
9. A computer-readable medium, characterized in that A computer program is stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.