Electrocardiogram monitoring method based on electromagnetic signal, monitoring model training method and device
By acquiring and compensating the initial electromagnetic echo signal, and using the ECG monitoring model to process the electromagnetic signal, the problem of information loss in contactless ECG monitoring is solved, and accurate twelve-lead ECG monitoring is achieved.
Patent Information
- Application Number
- CN202410968718.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-19
AI Technical Summary
The existing non-contact electrocardiogram monitoring methods using electromagnetic signals cannot accurately capture the heart's movement information, and a large amount of electrocardiogram semantic information will be lost during the mechanical-electric mode conversion.
By obtaining the initial electromagnetic echo signal, using the signal phase difference for compensation, the three-dimensional data of the target motion timing are obtained, and the encoder and decoder of the ECG monitoring model are used to process the multi-scale timing characteristics to generate a twelve-lead ECG image.
It realizes accurate twelve-lead ECG monitoring of the heart without contact, avoiding information loss and improving the accuracy and convenience of monitoring.
Smart Images

Figure CN118512183B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of electromagnetic sensing technology, and more specifically to an electrocardiogram monitoring method based on electromagnetic signals, a monitoring model training method and a device. Background Art
[0002] The electrocardiogram is one of the important biomedical images currently. Based on the electrocardiogram obtained through monitoring, using the electrocardiogram as an intermediate result can help make a targeted diagnosis of the target. Twelve-lead electrocardiogram monitoring is one of the important tools currently used to monitor the electrocardiogram of a target. The existing technology usually sticks sticky electrodes on the target, and then connects the electrodes and the monitoring device together through a wired connection to monitor the electrocardiogram of the target. Furthermore, scientists use electromagnetic signals to perform non-contact electrocardiogram monitoring of the target. However, the results obtained by the existing non-contact electrocardiogram monitoring of the target using electromagnetic signals have a large error compared with the results of wired electrocardiogram monitoring.
[0003] In the process of realizing the above-mentioned inventive concept, the inventors found that: in the related technology, since it is impossible to capture the movement information of the heart in space and can only extract the information of a single electromagnetic signal in time, it is impossible to obtain accurate movement information of the heart. In addition, in the process of converting the electromagnetic echo signal into the mechanical-electrical mode of the heart, a large amount of ECG semantic information will be lost in the conversion process due to the complexity of modeling and solving technical problems. Summary of the invention
[0004] In view of the above problems, the present invention provides an electrocardiogram monitoring method based on electromagnetic signals, a monitoring model training method and a device.
[0005] According to a first aspect of the present invention, a method for electrocardiogram monitoring based on electromagnetic signals is provided, characterized in that it includes: acquiring an initial electromagnetic echo signal, wherein the initial electromagnetic echo signal represents a plurality of spatially overlapped signals corresponding to the motion state of a target within a predetermined time period; compensating the initial electromagnetic echo signal according to a signal phase difference corresponding to the initial electromagnetic echo signal to obtain three-dimensional target motion time series data, wherein the three-dimensional target motion time series data includes time-series spatial motion information obtained by mapping the time-series motion state of the target in three-dimensional space; processing the three-dimensional target motion time series data using an electrocardiogram signal encoder of an electrocardiogram monitoring model to obtain multi-scale time series features, wherein the multi-scale time series features represent the time series information of the target mechanical motion at least two scales; processing the multi-scale time series features using an electrocardiogram signal decoder of the electrocardiogram monitoring model to obtain a twelve-lead electrocardiogram image, wherein the twelve-lead electrocardiogram image is used to represent the change in the motion state of the target within a predetermined time.
[0006] According to an embodiment of the present invention, the initial electromagnetic echo signal is compensated according to the signal phase difference corresponding to the initial electromagnetic echo signal to obtain target motion time series three-dimensional data, including: obtaining the signal phase difference based on the position information of the antenna receiving array and the bandwidth of the initial electromagnetic echo signal; performing phase compensation on the initial electromagnetic echo signal according to the signal phase difference to obtain the target electromagnetic echo signal, wherein the target electromagnetic echo signal includes a plurality of signals spatially separated from each other; and filtering the target electromagnetic echo signal to obtain the target motion time series three-dimensional data.
[0007] According to an embodiment of the present invention, a target electromagnetic echo signal includes an electromagnetic echo signal corresponding to each spatial point in three-dimensional space; filtering is performed on the target electromagnetic echo signal to obtain target motion time series three-dimensional data, including: splitting the electromagnetic echo signal corresponding to each spatial point to obtain multi-channel electromagnetic echo data corresponding to each spatial point, wherein the multi-channel electromagnetic echo data includes real channel electromagnetic echo data, imaginary channel electromagnetic echo data and phase channel electromagnetic echo data; performing time series difference processing on the multi-channel electromagnetic echo data corresponding to each spatial point to obtain multi-channel motion time series data corresponding to each spatial point; and obtaining target motion time series three-dimensional data based on the multi-channel motion time series data corresponding to all spatial points in three-dimensional space.
[0008] According to an embodiment of the present invention, an ECG signal encoder includes a three-dimensional space encoder and a timing encoder; the ECG signal encoder of the ECG monitoring model is used to process the target motion timing three-dimensional data to obtain multi-scale timing features, including: inputting the target motion timing three-dimensional data into the three-dimensional space encoder for encoding to obtain a first spatial feature, wherein the first spatial feature is a feature vector of a predetermined dimension; inputting the first spatial feature into the timing encoder for processing to obtain a multi-scale timing feature.
[0009] According to an embodiment of the present invention, a time series encoder includes a first convolution layer, a second convolution layer and a self-attention layer; the first spatial feature is input into the time series encoder for processing to obtain a multi-scale time series feature, including: the first spatial feature is input into the first convolution layer for downsampling processing to obtain a first convolution scale time series feature; the first convolution scale time series feature is input into the second convolution layer for downsampling processing to obtain a second convolution scale time series feature; the first convolution scale time series feature and the second convolution scale time series feature are input into the self-attention layer for enhancement processing to obtain a multi-scale time series feature.
[0010] According to an embodiment of the present invention, multi-scale time series features include first-scale time series features and second-scale time series features; the multi-scale time series features are processed using an ECG signal decoder of an ECG monitoring model to obtain a twelve-lead ECG image, including: inputting the second-scale time series features into the ECG signal decoder for deconvolution processing to obtain a first intermediate-scale time series feature; inputting the first-scale time series features into the ECG signal decoder and splicing them with the first intermediate-scale time series features to obtain a second intermediate-scale time series feature, wherein the dimension of the first-scale time series feature is consistent with the dimension of the first intermediate-scale time series feature; deconvolution processing is performed on the second intermediate-scale time series features to obtain a twelve-lead ECG image.
[0011] The second aspect of the present invention provides a training method for an ECG monitoring model, characterized in that the ECG monitoring model includes an ECG signal encoder and an ECG signal decoder, including: obtaining a first sample data set, wherein the first sample data set includes training time series three-dimensional data and twelve-lead ECG single-modality label data; inputting the training time series three-dimensional data into the ECG signal encoder to be trained for processing to obtain a first multi-scale sample time series feature, wherein the first multi-scale sample time series feature represents the time series information of the target mechanical motion at at least two scales; inputting the first multi-scale training time series feature into the ECG signal decoder to be trained for processing to obtain a first reconstructed twelve-lead ECG data; based on a loss function, obtaining a first loss value according to the first reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality training data; according to the first loss value, obtaining a trained ECG monitoring model by adjusting the model parameters of the ECG monitoring model.
[0012] According to an embodiment of the present invention, a trained ECG monitoring model is obtained by adjusting the model parameters of the ECG monitoring model according to a first loss value, including: according to the first loss value, fixing the model parameters of the ECG signal encoder of the ECG monitoring model, adjusting the model parameters of the ECG signal decoder of the ECG monitoring model, and obtaining an intermediate model; inputting a first sample data set into the intermediate model, and outputting a second reconstructed twelve-lead ECG data; based on a loss function, obtaining a second loss value according to the second reconstructed twelve-lead ECG data and twelve-lead ECG single-modality label data; based on the second loss value, fixing the model parameters of the ECG signal decoder of the ECG monitoring model, adjusting the model parameters of the ECG signal encoder of the ECG monitoring model, and obtaining a trained ECG monitoring model.
[0013] The third aspect of the present invention provides an electrocardiogram monitoring device based on electromagnetic signals, characterized in that it includes: a first acquisition module, used to acquire an initial electromagnetic echo signal, wherein the initial electromagnetic echo signal represents a plurality of spatially overlapped signals corresponding to the motion state of a target within a predetermined time period; a compensation module, used to compensate the initial electromagnetic echo signal according to a signal phase difference corresponding to the initial electromagnetic echo signal, to obtain three-dimensional target motion time series data, wherein the three-dimensional target motion time series data includes time-series spatial motion information obtained by mapping the time-series motion state of the target in three-dimensional space; a first processing module, used to process the three-dimensional target motion time series data using an electrocardiogram signal encoder of an electrocardiogram monitoring model, to obtain multi-scale time series features, wherein the multi-scale time series features represent the time series information of the target mechanical motion at least two scales; a second processing module, used to process the multi-scale time series features using an electrocardiogram signal decoder of the electrocardiogram monitoring model, to obtain a twelve-lead electrocardiogram image, wherein the twelve-lead electrocardiogram image is used to represent the change in the motion state of the target within a predetermined time.
[0014] The fourth aspect of the present invention provides a training device for an ECG monitoring model, characterized in that the ECG monitoring model includes an ECG signal encoder and an ECG signal decoder, including: a second acquisition module, used to acquire a first sample data set, wherein the first sample data set includes training time series three-dimensional data and twelve-lead ECG single-modality label data; an encoding module, used to input the training time series three-dimensional data into the ECG signal encoder to be trained for processing, and obtain a first multi-scale sample time series feature, wherein the first multi-scale sample time series feature represents the time series information of the target mechanical movement at at least two scales; a decoding module, used to input the first multi-scale training time series feature into the ECG signal decoder to be trained for processing, and obtain a first reconstructed twelve-lead ECG data; a loss module, used to obtain a first loss value based on the loss function according to the first reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality training data; a training module, used to obtain a trained ECG monitoring model by adjusting the model parameters of the ECG monitoring model according to the first loss value.
[0015] A fifth aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.
[0016] The sixth aspect of the present invention also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above method.
[0017] The seventh aspect of the present invention also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0018] According to the electrocardiogram monitoring method, monitoring model training method and device based on electromagnetic signals of the present invention, a millimeter wave radar is used to transmit multiple electromagnetic signals to a target and receive multiple electromagnetic echo signals to form initial electromagnetic echo signals that are spatially overlapped with each other and contain the motion state of the target within a predetermined time period. The initial electromagnetic echo signals are compensated according to the signal phase difference corresponding to the initial electromagnetic echo signals to separate the spatially overlapped initial electromagnetic echo signals to obtain three-dimensional target motion time series data in three-dimensional space. The three-dimensional target motion time series data includes the overall mechanical motion information of the target in the three-dimensional space and the motion changes of the target over time within the monitoring period, so that the data processed by the electrocardiogram monitoring model can reflect the state information of the mechanical motion of the target in time and space as a whole, so that a complete and comprehensive monitoring result can be finally obtained. Then, the target motion time series three-dimensional data is processed using the ECG monitoring model. The target motion time series three-dimensional data is first processed by the ECG signal encoder of the ECG monitoring model to obtain multi-scale time series features containing time series information of the target mechanical motion at at least two scales. The multi-scale time series features are then processed by the ECG signal decoder of the ECG monitoring model to finally obtain a twelve-lead ECG image containing the target motion state changes within a predetermined time. The twelve-lead ECG monitoring of the target of the monitored object is achieved by using electromagnetic signals without contacting the monitored object, obtaining accurate and complete ECG monitoring results, improving the monitoring experience of the monitored object, and making it easier for professionals and monitored objects to operate conveniently.
[0019] According to the embodiments of the present invention, since an initial ECG monitoring model is built according to a deep learning network architecture and a discrete variational autoencoder network, and then the built ECG monitoring model is trained to obtain an ECG monitoring model including a trained ECG signal encoder and an ECG signal decoder, when processing the target motion time series three-dimensional data, the trained ECG monitoring model is used for processing, and the ECG monitoring model is used to further process the spatial and temporal features in the target motion time series three-dimensional data in a data model-driven manner, thereby solving the problem of converting the heart from a mechanical motion form to an electrical domain signal form, avoiding the loss of ECG semantic information during the conversion process, and obtaining information including all ECG semantic information and mechanical motion state characteristics of the heart within the monitoring period. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0021] Figure 1A diagram showing an application scenario of an electrocardiogram monitoring method based on electromagnetic signals according to an embodiment of the present invention;
[0022] Figure 2 A flow chart showing a method for electrocardiogram monitoring based on electromagnetic signals according to an embodiment of the present invention is shown;
[0023] Figure 3 A flow chart showing a method for training an electrocardiogram monitoring model according to an embodiment of the present invention is shown;
[0024] Figure 4a A top view showing the energy spatial distribution of electromagnetic echo signals at different positions after spatial separation according to an embodiment of the present invention is shown;
[0025] Figure 4b A side view showing the energy spatial distribution of electromagnetic echo signals at different positions after spatial separation according to an embodiment of the present invention;
[0026] Figure 4c A front view showing the energy spatial distribution of electromagnetic echo signals at different positions after spatial separation according to an embodiment of the present invention;
[0027] Figure 5 A schematic diagram showing a waveform of a continuously changing multi-channel electromagnetic echo data in a complex plane according to an embodiment of the present invention;
[0028] Figure 6 A schematic diagram showing a network architecture of a discrete variational autoencoder according to an embodiment of the present invention is shown;
[0029] Figure 7 A schematic diagram showing a network architecture of an electrocardiogram monitoring model according to an embodiment of the present invention;
[0030] Figure 8a A schematic diagram showing a comparison between a non-contact first-lead ECG monitoring result and a synchronously monitored contact first-lead ECG waveform according to an embodiment of the present invention;
[0031] Figure 8b A schematic diagram showing a comparison between a non-contact second lead ECG monitoring result and a synchronously monitored contact second lead ECG waveform according to an embodiment of the present invention;
[0032] Figure 8c A schematic diagram showing a comparison between a non-contact third-lead ECG monitoring result and a synchronously monitored contact third-lead ECG waveform according to an embodiment of the present invention;
[0033] Figure 8d A schematic diagram showing a comparison between a non-contact pressurized unipolar right upper limb lead ECG monitoring result and a synchronously monitored contact pressurized unipolar right upper limb lead ECG waveform according to an embodiment of the present invention;
[0034] Figure 8e A schematic diagram showing a comparison between a non-contact pressurized unipolar left upper limb lead ECG monitoring result and a synchronously monitored contact pressurized unipolar left upper limb lead ECG waveform according to an embodiment of the present invention;
[0035] Figure 8f A schematic diagram showing a comparison between the non-contact pressurized unipolar left lower limb lead ECG monitoring result and the synchronously monitored contact pressurized unipolar left lower limb lead ECG waveform according to an embodiment of the present invention;
[0036] Figure 8g A schematic diagram showing a comparison between a non-contact first chest lead ECG monitoring result and a synchronously monitored contact first chest lead ECG waveform according to an embodiment of the present invention;
[0037] Figure 8h A schematic diagram showing a comparison between a non-contact second chest lead ECG monitoring result and a synchronously monitored contact second chest lead ECG waveform according to an embodiment of the present invention;
[0038] Figure 8i A schematic diagram showing a comparison between a non-contact third chest lead ECG monitoring result and a synchronously monitored contact third chest lead ECG waveform according to an embodiment of the present invention;
[0039] Figure 8j A schematic diagram showing a comparison between a non-contact fourth chest lead ECG monitoring result and a synchronously monitored contact fourth chest lead ECG waveform according to an embodiment of the present invention;
[0040] Figure 8k A schematic diagram showing a comparison between a non-contact fifth chest lead ECG monitoring result and a synchronously monitored contact fifth chest lead ECG waveform according to an embodiment of the present invention;
[0041] Figure 8l A schematic diagram showing a comparison between a non-contact sixth chest lead ECG monitoring result and a synchronously monitored contact sixth chest lead ECG waveform according to an embodiment of the present invention;
[0042] Figure 9a A schematic diagram showing the mean square error distribution of the non-contact twelve-lead electrocardiogram monitoring result compared with the contact measurement result according to an embodiment of the present invention;
[0043] Figure 9b A schematic diagram showing waveform correlation of a non-contact twelve-lead electrocardiogram monitoring result compared with a contact measurement result according to an embodiment of the present invention;
[0044] Fig.10 A structural block diagram of an electrocardiogram monitoring device based on electromagnetic signals according to an embodiment of the present invention is shown;
[0045] Fig.11 A structural block diagram of a training device for an electrocardiogram monitoring model according to an embodiment of the present invention is shown;
[0046] Fig.12 A block diagram of an electronic device for an electrocardiogram monitoring method based on electromagnetic signals according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0047] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.
[0048] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0049] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0050] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0051] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0052] Electrocardiogram is one of the important biomedical images at present. According to the electrocardiogram of the target obtained by monitoring, the electrocardiogram can be used as an intermediate result to help make targeted diagnosis or preventive measures for the target. There are many existing tools for ECG monitoring of targets, such as single-lead ECG monitoring and twelve-lead ECG monitoring. Among them, twelve-lead ECG monitoring is one of the important tools currently used for ECG monitoring of targets, and it is also one of the gold standards for helping to make a diagnosis. The existing technology usually sticks sticky electrodes on the target, and then connects the electrodes and the monitoring device together through a wired connection to monitor the target. However, the monitoring method of sticky electrodes and wired connections and the manual operation during the monitoring process result in contact twelve-lead ECG monitoring being mainly used in scenarios of periodic physical examination prevention or medical treatment after physical discomfort, so that it is difficult to capture instantaneous abnormalities of the heart and miss the best time. Furthermore, because electromagnetic signals can measure mechanical movement passively without contact, without line of sight, and without induction, scientists use electromagnetic signals to perform non-contact ECG monitoring on targets.
[0053] However, the results of the existing non-contact ECG monitoring using electromagnetic signals have a large error with the results of contact ECG monitoring, which makes it difficult to use non-contact electromagnetic signals to monitor the target in ECG monitoring. During the research and development process, the researchers found that the relevant technology could not capture the motion information of the heart in space and could only extract the information of a single electromagnetic signal in time, resulting in the inability to obtain accurate heart motion information. In addition, in the process of converting the electromagnetic echo signal to the mechanical-electrical mode of the heart, a large amount of ECG semantic information will be lost during the conversion process due to the complexity of modeling and solving technical problems.
[0054] In view of this, an embodiment of the present invention provides a method for electrocardiogram monitoring based on electromagnetic signals, characterized in that it includes: acquiring an initial electromagnetic echo signal, wherein the initial electromagnetic echo signal represents a plurality of spatially overlapped signals corresponding to the motion state of a target within a predetermined time period; compensating the initial electromagnetic echo signal according to a signal phase difference corresponding to the initial electromagnetic echo signal to obtain three-dimensional target motion time series data, wherein the three-dimensional target motion time series data includes time-series spatial motion information obtained by mapping the time-series motion state of the target in three-dimensional space; processing the three-dimensional target motion time series data using an encoder of an electrocardiogram monitoring model to obtain multi-scale time series features, wherein the multi-scale time series features represent the time series information of the target mechanical motion at least two scales; processing the multi-scale time series features using a decoder of the electrocardiogram monitoring model to obtain a twelve-lead electrocardiogram image, wherein the twelve-lead electrocardiogram image is used to represent the change in the motion state of the target within a predetermined time.
[0055] Figure 1 The application scenario diagram of the electrocardiogram monitoring method based on electromagnetic signals according to an embodiment of the present invention is schematically shown.
[0056] like Figure 1 As shown, the application scenario according to this embodiment may include a first millimeter wave radar device 101, a target object 102, and a receiver 103. The first radar device 101 is used to send an electromagnetic wave signal to the target object 102.
[0057] The user may use the first radar device 101 to interact with the target object 102 and the receiver 103 to receive or send signals, etc.
[0058] The receiver 103 may be a receiver for receiving various electromagnetic echo signals, for example, receiving and processing the electromagnetic signal sent by the first radar device 101 (for example only). The receiver 103 may analyze and process the received electromagnetic echo signal and other data, and feed back the processing result (for example, a web page, information, or data obtained or generated according to a user request) to the terminal device.
[0059] It should be noted that the electrocardiogram monitoring method based on electromagnetic signals provided in the embodiment of the present invention can generally be performed by the receiver 103. Accordingly, the electrocardiogram monitoring device based on electromagnetic signals provided in the embodiment of the present invention can generally be set in the receiver 103. The electrocardiogram monitoring method based on electromagnetic signals provided in the embodiment of the present invention can also be performed by a receiver or a receiver cluster that is different from the receiver 103 and can communicate with the first radar device 101 and / or the receiver 103. Accordingly, the electrocardiogram monitoring device based on electromagnetic signals provided in the embodiment of the present invention can also be set in a receiver or a receiver cluster that is different from the receiver 103 and can communicate with the first radar device 101 and / or the receiver 103.
[0060] It should be understood that Figure 1 The number of the first radar device, the target object and the receiver in the embodiment is only for illustration. According to the implementation requirements, there may be any number of radar devices, observation objects and receivers.
[0061] The following will be based on Figure 1 The scene described by Figure 2~Figure 9b The method for electrocardiogram monitoring based on electromagnetic signals of the disclosed embodiment is described in detail.
[0062] Figure 2 The flowchart of the method for electrocardiogram monitoring based on electromagnetic signals according to an embodiment of the present invention is schematically shown.
[0063] like Figure 2 As shown, the method for electrocardiogram monitoring based on electromagnetic signals of this embodiment includes operations S210 to S240.
[0064] In operation S210, an initial electromagnetic echo signal is acquired.
[0065] According to an embodiment of the present invention, the initial electromagnetic echo signal represents a plurality of spatially overlapped signals corresponding to the motion state of the target within a predetermined period of time.
[0066] According to an embodiment of the present invention, a millimeter wave radar may be used to transmit an electromagnetic signal to a target and receive an initial electromagnetic echo signal.
[0067] According to an embodiment of the present invention, the predetermined period of time may be characterized as a period of time during which the motion state of the target is detected using the millimeter wave radar, and the motion state may be characterized as a state of mechanical activity of the target.
[0068] For example, a millimeter wave radar may be used to transmit an electromagnetic signal to the heart and receive an initial electromagnetic echo signal. The initial electromagnetic echo signal received by the radar antenna array includes information about the mechanical activity of the heart during the radar detection period.
[0069] According to an embodiment of the present invention, the electrocardiogram monitoring based on electromagnetic signals of the present invention can monitor the target based on non-contact twelve-lead electrocardiogram monitoring. The millimeter-wave radar used for non-contact twelve-lead electrocardiogram monitoring has three transmitting antennas, which can transmit three electromagnetic signals to the target. The antenna receiving array of the millimeter-wave radar includes four receivers, which can receive electromagnetic echo signals from four different directions.
[0070] According to an embodiment of the present invention, since the millimeter wave radar for contactless twelve-lead ECG monitoring has three transmitting antennas and four receivers, the antenna receiving array of the millimeter wave radar can receive 3*4 electromagnetic echo signals, that is, 12 electromagnetic echo signals.
[0071] According to an embodiment of the present invention, 3*4 electromagnetic echo signals form a 3*4 matrix signal, ie, an initial electromagnetic echo signal.
[0072] According to an embodiment of the present invention, since each receiver is located at a different position, the received initial electromagnetic echo signal is a plurality of electromagnetic echo signals spatially overlapped with each other.
[0073] In operation S220, the initial electromagnetic echo signal is compensated according to a signal phase difference corresponding to the initial electromagnetic echo signal to obtain target motion time series three-dimensional data.
[0074] According to an embodiment of the present invention, the target motion temporal three-dimensional data includes temporal spatial motion information obtained by mapping the temporal motion state of the target in the three-dimensional space.
[0075] According to an embodiment of the present invention, phase compensation is performed on the initial electromagnetic echo signal based on the signal phase difference corresponding to each electromagnetic echo signal in the initial electromagnetic echo signal, thereby spatially separating multiple electromagnetic echo signals that were originally spatially overlapped with each other to obtain target motion time series three-dimensional data.
[0076] According to an embodiment of the present invention, the three-dimensional space may be characterized as a three-dimensional space where the target is located. The three-dimensional space may be a space of a size of 15*15*15, and may contain 15*15*15 three-dimensional space points.
[0077] According to an embodiment of the present invention, the target motion time series three-dimensional data includes both the motion information of the target in the three-dimensional space during the monitoring period and the motion information of the target changing with time during the monitoring period.
[0078] In operation S230, the target motion time series three-dimensional data is processed using an ECG signal encoder of an ECG monitoring model to obtain multi-scale time series features.
[0079] According to an embodiment of the present invention, the multi-scale temporal features characterize the temporal information of the target mechanical motion at least at two scales.
[0080] According to an embodiment of the present invention, the multi-scale time series feature may include time series information of target mechanical motions at two different scales, and the time series information of target mechanical motions at two different scales is feature information obtained after being processed by an encoder of an electrocardiogram monitoring model.
[0081] In operation S240 , the multi-scale time series features are processed using an ECG signal decoder of the ECG monitoring model to obtain a twelve-lead ECG image.
[0082] According to an embodiment of the present invention, the twelve-lead electrocardiogram image is used to characterize the change of the motion state of the target within a predetermined time.
[0083] According to an embodiment of the present invention, after processing the multi-scale time series features using the decoder of the ECG monitoring model, a twelve-lead ECG image can be obtained, which may also include twelve-lead ECG data results. The twelve-lead ECG image may include semantic information and motion information of the target state.
[0084] According to an embodiment of the present invention, a trained ECG monitoring model is obtained by training the ECG monitoring model to be trained. The ECG monitoring model includes an ECG signal decoder and an ECG signal encoder. The ECG signal decoder can downsample and extract features of the target motion time series three-dimensional data to obtain multi-scale time series features. The ECG signal encoder can process the multi-scale time series features obtained by the decoder.
[0085] According to an embodiment of the present invention, a trained ECG monitoring model is used to process target motion time series three-dimensional data including target motion state, thereby outputting twelve-lead ECG measurement results at consecutive moments within a monitoring period, thereby completing contactless twelve-lead ECG monitoring.
[0086] For example, non-contact twelve-lead ECG monitoring is performed on the target object, an electromagnetic signal is emitted to the heart of the target object, an initial electromagnetic echo signal is received, and the initial electromagnetic echo signal is processed to obtain target motion time series three-dimensional data including the heart motion of the target object, and then the target motion time series three-dimensional data including the heart motion of the target object is processed using a trained ECG monitoring model to obtain a twelve-lead ECG image of the target object, so that scientists or professionals in related fields can identify the twelve-lead ECG image of the target object and make a judgment appropriate for the target object.
[0087] According to an embodiment of the present invention, by using a millimeter wave radar to transmit multiple electromagnetic signals to a target and receiving multiple electromagnetic echo signals, an initial electromagnetic echo signal containing the spatial overlap of the target's motion state within a predetermined period of time is formed, and the initial electromagnetic echo signal is compensated according to the signal phase difference corresponding to the initial electromagnetic echo signal, so that the initial electromagnetic echo signal that is spatially overlapped is separated, and the target motion time series three-dimensional data in three-dimensional space is obtained, and the target motion time series three-dimensional data contains the overall mechanical motion information of the target in three-dimensional space and the motion change of the target over time within the monitoring period, so that the data processed by the electrocardiogram monitoring model can reflect the state information of the mechanical motion of the target in time and space as a whole, so as to finally obtain a complete and comprehensive monitoring result. Then, the target motion time series three-dimensional data is processed by the electrocardiogram monitoring model, and the target motion time series three-dimensional data is first processed by the electrocardiogram signal encoder of the electrocardiogram monitoring model to obtain a multi-scale time series feature containing the time series information of the target mechanical motion of at least two scales, and then the electrocardiogram signal decoder of the electrocardiogram monitoring model is used to process the multi-scale time series feature, and finally a twelve-lead electrocardiogram image containing the motion state change of the target within a predetermined time is obtained. It realizes the use of electromagnetic signals to perform twelve-lead ECG monitoring on the target part of the monitored object without contacting the monitored object, so as to obtain accurate and complete ECG monitoring results, improve the monitoring experience of the monitored object, and make it easier for professionals and monitored objects to operate conveniently.
[0088] According to the embodiments of the present invention, since an initial ECG monitoring model is built according to a deep learning network architecture and a discrete variational autoencoder network, and then the built ECG monitoring model is trained to obtain an ECG monitoring model including a trained ECG signal encoder and an ECG signal decoder, when processing the target motion time series three-dimensional data, the trained ECG monitoring model is used for processing, and the ECG monitoring model is used to further process the spatial and temporal features in the target motion time series three-dimensional data in a data model-driven manner, thereby solving the problem of converting the heart from a mechanical motion form to an electrical domain signal form, avoiding the loss of ECG semantic information during the conversion process, and obtaining information including all ECG semantic information and mechanical motion state characteristics of the heart within the monitoring period.
[0089] It should be noted that the twelve-lead ECG image obtained by the present invention is only an intermediate result, and the diagnosis result or health condition cannot be directly derived from the twelve-lead ECG image obtained by the method of the present invention.
[0090] According to an embodiment of the present invention, the initial electromagnetic echo signal is compensated according to the signal phase difference corresponding to the initial electromagnetic echo signal to obtain the target motion time series three-dimensional data, including:
[0091] According to an embodiment of the present invention, a signal phase difference is obtained based on the position information of the antenna receiving array and the bandwidth of the initial electromagnetic echo signal.
[0092] According to an embodiment of the present invention, a spatial phase shift steering matrix is constructed based on the position information of the antenna receiving array of the millimeter wave radar and the bandwidth of the initial electromagnetic echo signal. The spatial phase shift steering matrix is a complex matrix. The spatial phase shift steering matrix includes channel amplitude and phase difference. Each element in the spatial phase shift steering matrix can be represented as a channel gain from a certain receiving antenna to a certain transmitting antenna. The phase of each element determines the relative delay or advance of the electromagnetic echo signal, so that the signal phase difference can be determined based on the phase of each element.
[0093] According to an embodiment of the present invention, phase compensation is performed on the initial electromagnetic echo signal according to the signal phase difference to obtain a target electromagnetic echo signal.
[0094] According to an embodiment of the present invention, the target electromagnetic echo signal includes a plurality of signals that are spatially separated from each other.
[0095] According to an embodiment of the present invention, the sampling frequency of the target electromagnetic echo signal is 100 Hz.
[0096] According to an embodiment of the present invention, the initial electromagnetic echo signal is phase compensated according to the signal phase difference included in the spatial phase shift steering matrix, so that multiple electromagnetic echo signals that are spatially overlapped with each other are spatially separated to obtain electromagnetic echo signals at different reflection positions, i.e., target electromagnetic echo signals.
[0097] According to an embodiment of the present invention, the target electromagnetic echo signal can be calculated according to formula (1).
[0098] (1);
[0099] Among them, y(p, t) can be represented as the target electromagnetic echo signal, t can be represented as the tth moment in the monitoring period, p can be represented as the reflection position of the electromagnetic signal in three-dimensional space, j can be represented as the imaginary part, h(p, t) can be represented as the change amplitude of the reflection position due to the mechanical movement of the target, d(p) can be represented as the sum of the distance from the electromagnetic signal to the reflection position and the distance from the electromagnetic echo signal reflected back from the reflection position, d mm (p, t) can be characterized as the distance change of the projection of the reflection position on the signal transmission path due to the mechanical movement of the target, It can be represented as the wavelength of the electromagnetic signal at time t, where the three-dimensional space includes 15*15*15 spatial points.
[0100] According to an embodiment of the present invention, the target electromagnetic echo signal is filtered to obtain target motion time series three-dimensional data.
[0101] According to an embodiment of the present invention, the target electromagnetic echo signal is filtered to filter out the target electromagnetic echo signal below a predetermined frequency threshold to obtain motion time series three-dimensional data corresponding to each spatial point in the three-dimensional space, and the motion time series three-dimensional data corresponding to each spatial point are combined to obtain target motion time series three-dimensional data containing the motion time series three-dimensional data corresponding to each spatial point.
[0102] According to an embodiment of the present invention, a spatial phase shift steering matrix including a signal phase difference for phase compensation is constructed based on the position information of the antenna receiving array and the bandwidth of the initial electromagnetic echo signal, and phase compensation is performed on the initial electromagnetic echo signals that are spatially overlapped with each other according to the signal phase difference in the spatial phase shift steering matrix, so that the electromagnetic echo signals are spatially separated from each other, so as to facilitate beamforming in space to obtain the target electromagnetic echo signal, and then the target electromagnetic echo signal is filtered to obtain the target motion time-series three-dimensional data, thereby realizing the spatial separation of the initial electromagnetic echo signal to obtain the electromagnetic echo signal of each reflection position in the three-dimensional space at each moment during the monitoring process, and through spatial separation, beamforming, filtering processing and other operations, the target motion time-series three-dimensional data includes complete spatial motion information and time-series motion information, thereby realizing the acquisition of complete cardiac mechanical motion data in space and time.
[0103] According to an embodiment of the present invention, the target electromagnetic echo signal includes an electromagnetic echo signal corresponding to each space point in the three-dimensional space.
[0104] According to an embodiment of the present invention, filtering is performed on the target electromagnetic echo signal to obtain target motion time series three-dimensional data, including:
[0105] According to an embodiment of the present invention, the electromagnetic echo signal corresponding to each spatial point is split to obtain multi-channel electromagnetic echo data corresponding to each spatial point.
[0106] According to an embodiment of the present invention, the multi-channel electromagnetic echo data includes real channel electromagnetic echo data, imaginary channel electromagnetic echo data and phase channel electromagnetic echo data.
[0107] According to an embodiment of the present invention, a multi-channel holographic mechanical motion measurement representation is constructed for the target electromagnetic echo signal corresponding to each spatial point to obtain real channel electromagnetic echo data, imaginary channel electromagnetic echo data and phase channel electromagnetic echo data.
[0108] According to an embodiment of the present invention, by constructing multi-channel electromagnetic echo data, factors such as frequency offset and amplitude change of the electromagnetic echo signal that may be caused by the mechanical movement of the target are avoided, thereby avoiding the accuracy and completeness of the twelve-lead ECG data obtained by monitoring, thereby improving the accuracy of the twelve-lead ECG data obtained by monitoring.
[0109] According to an embodiment of the present invention, time series difference processing is performed on multi-channel electromagnetic echo data corresponding to each spatial point to obtain multi-channel motion time series data corresponding to each spatial point.
[0110] According to an embodiment of the present invention, the time series difference of the multi-channel electromagnetic echo data corresponding to each spatial point can be calculated according to formula (2).
[0111] (2);
[0112] Among them, w k It can be represented as a filter template, y signal It can be represented as multi-channel motion timing data, k can be represented as the length of the filter template, It can be characterized as multi-channel electromagnetic echo data above a predetermined frequency threshold at the pth reflection position at time t.
[0113] According to an embodiment of the present invention, by performing time-series difference processing on multi-channel electromagnetic echo data corresponding to each spatial point, the obtained multi-channel motion time-series data all retain high-frequency data related to the target mechanical motion.
[0114] According to an embodiment of the present invention, the multi-channel motion time series data corresponding to each spatial point can be calculated according to formula (3).
[0115] (3);
[0116] Among them, x(p, t) can be represented as multi-channel motion time series data corresponding to each spatial point, It can be characterized as the real channel electromagnetic echo data of the pth reflection position at time t that is higher than a predetermined frequency threshold, It can be characterized as the imaginary channel electromagnetic echo data above the predetermined frequency threshold at the p-th reflection position at time t, It can be characterized as the phase channel electromagnetic echo data of the pth reflection position at time t that is higher than a predetermined frequency threshold, The multi-channel electromagnetic echo data above the predetermined frequency threshold of the p-th reflection position at time t includes real channel electromagnetic echo data above the predetermined frequency threshold, imaginary channel electromagnetic echo data above the predetermined frequency threshold and phase channel electromagnetic echo data above the predetermined frequency threshold.
[0117] According to an embodiment of the present invention, target motion time series three-dimensional data is obtained based on multi-channel motion time series data corresponding to all spatial points in the three-dimensional space.
[0118] According to an embodiment of the present invention, the target motion time series three-dimensional data can be characterized as data of all spatial points in the three-dimensional space at the current moment obtained by including multi-channel motion time series data corresponding to all spatial points, that is, data of a size of 3*15*15*15, wherein the target motion time series three-dimensional data can be represented by x(t).
[0119] According to an embodiment of the present invention, by splitting the electromagnetic echo signal corresponding to each spatial point, the real channel electromagnetic echo data, the imaginary channel electromagnetic echo data and the phase channel electromagnetic echo data corresponding to each spatial point are obtained, and then the multi-channel electromagnetic echo data are subjected to time difference processing to obtain multi-channel motion time series data corresponding to each spatial point above a predetermined frequency threshold, and finally, according to the multi-channel motion time series data corresponding to all spatial points in the three-dimensional space, the target motion time series three-dimensional data is obtained, thereby obtaining the multi-channel motion time series three-dimensional data of all reflection positions in the three-dimensional space within the monitoring period, without the need to screen multiple time series data and then select one time series data as the target data for subsequent operation processing, and by inputting the multi-channel motion time series three-dimensional data of all spatial points in the three-dimensional space obtained by the present invention into the electrocardiogram monitoring model for processing, complete and accurate twelve-lead electrocardiogram data can be measured, thereby improving the measurement accuracy.
[0120] According to an embodiment of the present invention, the ECG signal encoder of the ECG monitoring model is used to process the target motion time series three-dimensional data to obtain multi-scale time series features, including:
[0121] According to an embodiment of the present invention, the target motion time series three-dimensional data is input into a three-dimensional space encoder for encoding to obtain a first spatial feature.
[0122] According to an embodiment of the present invention, the first spatial feature is a feature vector of a predetermined dimension.
[0123] According to an embodiment of the present invention, the ECG monitoring model includes a three-dimensional space encoder, a time series encoder and an ECG signal decoder, and the three-dimensional space encoder and the time series encoder together constitute the ECG signal encoder of the ECG monitoring model.
[0124] According to an embodiment of the present invention, the target motion time series three-dimensional data is input into a three-dimensional space encoder in a pre-trained electrocardiogram monitoring model for encoding processing to obtain a first spatial feature. The predetermined dimension of the first spatial feature can be 64 dimensions, that is, the first spatial feature is a 64-dimensional feature vector.
[0125] According to an embodiment of the present invention, the target motion timing three-dimensional data is three-dimensional data. After the target motion timing three-dimensional data is input into a three-dimensional space encoder, the first spatial feature obtained is a one-dimensional vector, which can include 64 spatial features.
[0126] According to an embodiment of the present invention, the first spatial feature is input into a time series encoder for processing to obtain a multi-scale time series feature.
[0127] According to an embodiment of the present invention, the multi-scale temporal features may include temporal features of target motion temporal three-dimensional data.
[0128] According to an embodiment of the present invention, by inputting the target motion time series three-dimensional data into a three-dimensional space encoder for encoding, a first spatial feature is obtained, and then the first spatial feature is input into a time series encoder for processing to obtain a multi-scale time series feature, and the target motion time series three-dimensional data is processed using an ECG monitoring model to convert the three-dimensional data into one-dimensional data for processing. At the same time, the spatial features and time series features in the target motion time series three-dimensional data are extracted, so that the decoder can decode and process according to the multi-scale time series features, convert the mechanical motion form into an electrical domain signal form, and obtain a complete twelve-lead electrocardiogram. Furthermore, the pre-trained three-dimensional space encoder and time series encoder can process a large amount of data, that is, 3*15*15*15 three-dimensional data can be processed without processing part of the motion time series three-dimensional data obtained by screening, thereby obtaining a twelve-lead electrocardiogram with lost ECG semantics.
[0129] According to an embodiment of the present invention, the temporal encoder includes a first convolutional layer, a second convolutional layer and a self-attention layer.
[0130] According to an embodiment of the present invention, the first spatial feature is input into a temporal encoder for processing to obtain a multi-scale temporal feature, including:
[0131] According to an embodiment of the present invention, the first spatial feature is input into the first convolution layer for downsampling processing to obtain the first convolution scale temporal feature.
[0132] According to an embodiment of the present invention, the first spatial feature is input into the first convolution layer for downsampling processing, and the obtained first convolution scale time series feature is a feature that has been downsampled 4 times compared to the first spatial feature.
[0133] According to an embodiment of the present invention, the first convolutional scale time series feature is input into the second convolutional layer for downsampling processing to obtain the second convolutional scale time series feature.
[0134] According to an embodiment of the present invention, the first convolutional scale temporal feature is further input into the second convolutional layer for downsampling processing, so that the second convolutional scale temporal feature is obtained, which is a feature that has been downsampled 8 times compared to the first spatial feature.
[0135] According to an embodiment of the present invention, the first convolutional scale time series feature and the second convolutional scale time series feature are input into the self-attention layer for enhancement processing to obtain multi-scale time series features.
[0136] According to an embodiment of the present invention, feature enhancement processing is performed on the temporal features and spatial features in the first convolution scale temporal features and the second convolution scale temporal features through a self-attention layer, so that the decoder can perform deconvolution processing on the enhanced features.
[0137] According to an embodiment of the present invention, the first spatial feature is input into the first convolution layer for downsampling processing to obtain the first convolution scale time series feature, and then the first convolution scale time series feature is input into the second convolution layer for downsampling processing to obtain the second convolution scale time series feature, and finally the first convolution scale time series feature and the second convolution scale time series feature are input into the self-attention layer for enhancement processing to obtain multi-scale time series features, thereby realizing time series processing of the first spatial feature, performing two downsampling processes through the two convolution layers in the time encoder to obtain time series features of two different scales, thereby avoiding the loss of ECG semantic information in the process of converting the heart from the mechanical motion form to the electrical domain signal form, and constructing a mapping relationship between the mechanical motion form and the electrical domain signal form.
[0138] According to an embodiment of the present invention, the multi-scale time series features include first-scale time series features and second-scale time series features.
[0139] According to an embodiment of the present invention, the multi-scale time series features are processed using an ECG signal decoder of an ECG monitoring model to obtain a twelve-lead ECG image, including:
[0140] According to an embodiment of the present invention, the second-scale time series features are input into an electrocardiogram signal decoder for deconvolution processing to obtain first intermediate-scale time series features.
[0141] According to an embodiment of the present invention, the ECG signal decoder is obtained based on discrete variational autoencoder network training.
[0142] According to an embodiment of the present invention, the first-scale time series feature is input into an electrocardiogram signal decoder and concatenated with the first intermediate-scale time series feature to obtain a second intermediate-scale time series feature.
[0143] According to an embodiment of the present invention, the dimension of the first-scale time series feature is consistent with the dimension of the first intermediate-scale time series feature.
[0144] According to an embodiment of the present invention, the first-scale time series feature and the first intermediate-scale time series feature are concatenated in dimension to obtain the second intermediate-scale time series feature.
[0145] For example, the size of the first-scale time series feature is 3*5, the size of the first intermediate-scale time series feature is 3*5, and the size of the second intermediate-scale time series feature obtained after splicing is 6*5.
[0146] According to an embodiment of the present invention, a deconvolution process is performed on the second intermediate-scale time series features to obtain a twelve-lead electrocardiogram image.
[0147] According to an embodiment of the present invention, the twelve-lead ECG image may include ECG semantic information and motion state information of the target during the monitoring period.
[0148] For example, a twelve-lead electrocardiogram image may include semantic information and motion state information of the heart of the monitored subject.
[0149] According to an embodiment of the present invention, by inputting the second-scale time series features into an ECG signal decoder for deconvolution processing, a first intermediate-scale time series features are obtained, the first-scale time series features are input into the ECG signal decoder, and spliced with the first intermediate-scale time series features to obtain second intermediate-scale time series features, and the second intermediate-scale time series features are deconvoluted to obtain a twelve-lead ECG image, so as to achieve twelve-lead ECG monitoring of the target location of the monitored object and obtain accurate and complete ECG monitoring results, and the decoder of the pre-trained ECG monitoring model is used to perform deconvolution processing on the first-scale time series and the second-scale time series to restore the ECG signal containing ECG semantic information and mechanical motion state, so as to convert the heart from the mechanical motion form to the electrical domain signal form, and construct a mapping relationship between the mechanical motion form and the electrical domain signal form.
[0150] Figure 3 The flowchart of the training method of the electrocardiogram monitoring model according to the embodiment of the present invention is schematically shown.
[0151] like Figure 3 As shown, the training method of the electrocardiogram monitoring model of this embodiment includes operations S310 to S350.
[0152] According to an embodiment of the present invention, the ECG monitoring model includes an ECG signal encoder and an ECG signal decoder.
[0153] In operation S310 , a first sample data set is acquired.
[0154] According to an embodiment of the present invention, the first sample data set includes training time-series three-dimensional data and twelve-lead ECG single-modality label data.
[0155] According to an embodiment of the present invention, the twelve-lead ECG single-modality label data is used to verify the accuracy of the ECG data output by the ECG monitoring model to be trained and to calculate the loss value.
[0156] In operation S320, the training time series three-dimensional data is input into the ECG signal encoder to be trained for processing to obtain a first multi-scale sample time series feature.
[0157] According to an embodiment of the present invention, the first multi-scale sample time series feature represents time series information of the target mechanical motion at least at two scales.
[0158] According to an embodiment of the present invention, the ECG signal encoder to be trained includes a three-dimensional space encoder to be trained and a temporal encoder to be trained.
[0159] According to an embodiment of the present invention, the training time series three-dimensional data is input into the spatial three-dimensional encoder to be trained for encoding to obtain a 64-dimensional first sample space feature, which is a feature vector.
[0160] According to an embodiment of the present invention, the training time series three-dimensional data is represented as follows: .
[0161] Among them, x ’ t It can be represented as training time series three-dimensional data, R 3*15*15*15 It can be characterized as the size of the training time series three-dimensional data.
[0162] According to an embodiment of the present invention, the first sample space feature can be calculated according to formula (4).
[0163] (4);
[0164] Among them, h t It can be characterized as the first sample space feature, E Voxel It can be represented as a three-dimensional spatial encoder in the ECG signal encoder to be trained, R d It can be represented as a set of dimension d.
[0165] According to an embodiment of the present invention, a spatial three-dimensional encoder can process multi-dimensional data, thereby retaining complete data and avoiding data loss or incomplete output results caused by a small amount of data input.
[0166] According to an embodiment of the present invention, a first sample space feature is input into a time series encoder to be trained, and the time series encoder to be trained includes two convolution layers and a self-attention layer. The first sample space feature passes through the first convolution layer to obtain a first-scale training time series feature, and the first-scale training time series feature is obtained after 4 times downsampling compared to the first sample space feature. Then, the first-scale training time series feature passes through the second convolution layer to obtain a second-scale training time series feature, and the second-scale training time series feature is obtained after 8 times downsampling compared to the first sample space feature. Then, the first-scale training time series feature and the second-scale training time series feature are subjected to feature enhancement processing through the self-attention layer to obtain a first multi-scale sample time series feature, and the first multi-scale sample time series feature includes sample time series features of two different scales, and the two different-scale sample time series features are respectively a time series feature obtained after the first sample space feature is enhanced and a time series feature obtained after the second sample space feature is enhanced.
[0167] In operation S330, the first multi-scale training time series feature is input into the ECG signal decoder to be trained for processing to obtain first reconstructed twelve-lead ECG data.
[0168] According to an embodiment of the present invention, the ECG signal decoder is obtained based on a discrete variational autoencoder network.
[0169] According to an embodiment of the present invention, the first sample data set may include a twelve-lead electrocardiogram signal, which is downsampled to a frequency of 100 Hz and then divided into segment training electrocardiogram data with a length of 512 to obtain N segment training electrocardiogram data.
[0170] According to an embodiment of the present invention, a multi-scale discrete variational autoencoder network is constructed, and the discrete variational autoencoder network includes an encoder and a decoder. N fragment training ECG data are input into the encoder of the discrete variational autoencoder network, and first-scale training ECG data with a length of 128 are obtained through a first convolution layer. The first-scale training ECG data are then passed through a second convolution layer to obtain second-scale training ECG data with a length of 64. Then, a dictionary codebook with a size of 512 is constructed according to the first-scale training ECG data and the second-scale training ECG data. A dictionary representation of the time series representation corresponding to the scale training ECG data is obtained from the dictionary codebook to obtain first dictionary training ECG data and second dictionary training ECG data. Then, the first dictionary training ECG data and the second dictionary training ECG data are input into the decoder. The second dictionary training ECG data passes through a first deconvolution layer to obtain first intermediate training ECG data. The first intermediate training ECG data and the first dictionary training ECG data are spliced to obtain second intermediate training ECG data. The second intermediate training ECG data is input into the second convolution layer to obtain third reconstructed twelve-lead ECG data.
[0171] According to an embodiment of the present invention, when the third reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality label data satisfy formula (5), the discrete variational autoencoder network decoder is confirmed as the ECG signal decoder to be trained.
[0172] (5);
[0173] Among them, l rec It can be represented as the error threshold, S i It can be represented as 12-lead ECG single-modality label data. It can be characterized as the third reconstructed twelve-lead ECG data.
[0174] According to an embodiment of the present invention, when the error threshold is 0, the discrete variational autoencoder network decoder is confirmed as the electrocardiogram signal decoder to be trained.
[0175] According to an embodiment of the present invention, the first-scale training time series features and the second-scale training time series features are input into the ECG signal decoder to be trained, the second-scale training time series features are passed through a first deconvolution layer to obtain a first intermediate training scale time series features, and then the first-scale training time series features and the first intermediate training scale time series features are spliced to obtain a second intermediate training scale time series features, and the second intermediate training scale time series features are passed through a second deconvolution layer to output the first reconstructed twelve-lead ECG data.
[0176] In operation S340 , a first loss value is obtained based on the loss function and the first reconstructed 12-lead ECG data and the 12-lead ECG single-modality training data.
[0177] In operation S350, a trained ECG monitoring model is obtained by adjusting model parameters of the ECG monitoring model according to the first loss value.
[0178] According to an embodiment of the present invention, the model parameters of the ECG monitoring model are adjusted according to the first loss value, and then the adjusted ECG monitoring model can be error trained until correct and complete reconstructed twelve-lead ECG data that is consistent with the twelve-lead ECG single-modality training data is output, and the model at this time is determined to be a trained ECG monitoring model.
[0179] According to an embodiment of the present invention, a first sample data set including training time series three-dimensional data and twelve-lead ECG single-modal label data is obtained, and then the training time series three-dimensional data is input into the ECG signal encoder to be trained to obtain a first multi-scale sample time series feature, and then the first multi-scale sample time series feature is input into the ECG signal decoder to be trained for processing to obtain a first reconstructed twelve-lead ECG data, based on a loss function, according to the first reconstructed twelve-lead ECG data and the twelve-lead ECG single-modal training data, a first loss value is obtained, according to the first loss value, by adjusting the model parameters of the ECG monitoring model, a trained ECG monitoring model is obtained, and the training of the ECG monitoring model is realized, the loss value is obtained according to the loss function, and the encoder and decoder in the ECG monitoring model are repeatedly trained until a model that can accurately output a twelve-lead ECG is obtained, and a model for converting the heart from a mechanical motion form to an electrical domain signal form is constructed in a data-driven manner.
[0180] According to an embodiment of the present invention, according to the first loss value, a trained ECG monitoring model is obtained by adjusting the model parameters of the ECG monitoring model, including:
[0181] According to an embodiment of the present invention, based on the first loss value, the model parameters of the ECG signal encoder of the ECG monitoring model are fixed, and the model parameters of the ECG signal decoder of the ECG monitoring model are adjusted to obtain an intermediate model.
[0182] According to an embodiment of the present invention, the encoder and decoder may be separated for parameter adjustment. The model parameters of the encoder are first fixed, and the model parameters of the decoder are adjusted to obtain an intermediate model.
[0183] According to an embodiment of the present invention, the first sample data set is input into the intermediate model, and the second reconstructed twelve-lead ECG data is output.
[0184] According to an embodiment of the present invention, the first sample data set is then re-input into the intermediate model, and the training of the ECG signal encoder is determined to be completed based on the output of the second reconstructed twelve-lead ECG data.
[0185] According to an embodiment of the present invention, based on the loss function, a second loss value is obtained according to the second reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality label data.
[0186] According to an embodiment of the present invention, based on the second loss value, the model parameters of the ECG signal decoder of the ECG monitoring model are fixed, and the model parameters of the ECG signal encoder of the ECG monitoring model are adjusted to obtain a trained ECG monitoring model.
[0187] According to an embodiment of the present invention, based on the second loss value, the model parameters of the decoder are fixed, and the model parameters of the encoder are adjusted to obtain a trained electrocardiogram monitoring model.
[0188] According to an embodiment of the present invention, the model parameters of the decoder may be fixed first, and the model parameters of the encoder may be adjusted. Then, based on the loss value, the model parameters of the encoder may be fixed again, and the model parameters of the decoder may be adjusted to finally obtain a trained ECG monitoring model.
[0189] According to an embodiment of the present invention, the model parameters of the encoder and the decoder can also be adjusted simultaneously, and a trained ECG monitoring model can be obtained based on the obtained new loss value.
[0190] According to an embodiment of the present invention, based on the first loss value, the model parameters of the ECG signal encoder of the ECG monitoring model are first fixed, the model parameters of the ECG signal decoder are adjusted to obtain an intermediate model, and then the first sample data set is input into the intermediate model, the intermediate model is trained and verified, and the second reconstructed twelve-lead ECG data is output. Based on the loss function, the second reconstructed twelve-lead ECG data and the twelve-lead ECG single-modal label data are processed to obtain a second loss value, and then based on the second loss value, the model parameters of the ECG signal decoder are fixed, the model parameters of the ECG signal encoder are adjusted to obtain the target model, that is, the trained ECG monitoring model, which realizes targeted parameter adjustment of the decoder and encoder of the ECG monitoring model according to the loss value, respectively, and constructs a network model for converting cardiac mechanical data to electrical domain signals, thereby obtaining an ECG monitoring model that can output an accurate twelve-lead ECG.
[0191] Figure 4a A top view of the energy spatial distribution of electromagnetic echo signals at different positions after spatial separation according to an embodiment of the present invention is shown.
[0192] like Figure 4a As shown, after using the millimeter wave radar to transmit electromagnetic signals to the monitored object and receive the initial electromagnetic echo signals, the energy distribution of the spatially separated target electromagnetic echo signals at different spatial points in the three-dimensional space is obtained after phase compensation of the initial electromagnetic echo signals that are spatially overlapped with each other. It can be seen from the top view that the energy is mainly distributed on the left side.
[0193] Figure 4b A side view showing the spatial distribution of energy of electromagnetic echo signals at different positions after spatial separation according to an embodiment of the present invention is shown.
[0194] like Figure 4b As shown, after using the millimeter wave radar to transmit electromagnetic signals to the monitored object and receive the initial electromagnetic echo signals, the energy distribution of the spatially separated target electromagnetic echo signals at different spatial points in the three-dimensional space is obtained after phase compensation of the initial electromagnetic echo signals that are spatially overlapped with each other. It can be seen from the side view that the energy is mainly distributed in the middle.
[0195] Figure 4c A front view showing the energy spatial distribution of electromagnetic echo signals at different positions after spatial separation according to an embodiment of the present invention is shown.
[0196] like Figure 4c As shown, after using the millimeter wave radar to transmit electromagnetic signals to the monitored object and receive the initial electromagnetic echo signals, the energy distribution of the spatially separated target electromagnetic echo signals at different spatial points in the three-dimensional space is obtained after phase compensation of the initial electromagnetic echo signals that are spatially overlapped with each other. It can be seen from the front view that the energy is mainly distributed in the middle and the monitoring range is large.
[0197] According to the embodiment of the present invention, it can be seen from the top view, the side view and the bottom view that the state of the cardiac mechanical movement is positively correlated with the energy intensity. The more intense the cardiac mechanical movement, the stronger the energy intensity.
[0198] Figure 5 A schematic diagram showing a continuously changing waveform of multi-channel electromagnetic echo data in a complex plane according to an embodiment of the present invention is shown.
[0199] like Figure 5As shown, the continuously changing waveform of the multi-channel electromagnetic echo data in the complex plane is shown. The multi-channel electromagnetic echo data includes real channel electromagnetic echo data, imaginary channel electromagnetic echo data and phase channel electromagnetic echo data. The horizontal axis I can be represented as the imaginary part, and the vertical axis Q can be represented as the real part. From each dot in the figure, the changing waveform of the multi-channel electromagnetic echo data in the complex plane can be seen, and the angle of each dot represents the phase.
[0200] Figure 6 A schematic diagram of a network architecture of a discrete variational autoencoder according to an embodiment of the present invention is shown.
[0201] like Figure 6 As shown, the network architecture of the discrete variational autoencoder is shown. The network architecture of the discrete variational autoencoder includes an encoder, a codebook and a decoder. The ECG signal can be characterized as twelve-lead ECG single-modal training data. The reconstructed signal can be characterized as the third reconstructed twelve-lead ECG data. The twelve-lead ECG single-modal training data is downsampled to a frequency of 100 Hz, and then divided into segment training ECG data with a length of 512 to obtain N segment training ECG data S i Then, the N segments of training ECG data are input into the encoder of the discrete variational autoencoder network, and the first scale training ECG data S with a length of 128 is obtained after the first convolution layer. i (b) , and then the first scale training ECG data is passed through the second convolution layer to obtain the second scale training ECG data S with a length of 64 i (t) Then, a dictionary codebook of size 512 is constructed based on the first scale training ECG data and the second scale training ECG data, and a dictionary representation of the time series representation corresponding to the scale training ECG data is obtained from the dictionary codebook to obtain the first dictionary training ECG data C i (b) and the second dictionary training ECG data C i (b) , then input the first dictionary training ECG data and the second dictionary training ECG data into the decoder, the second dictionary training ECG data passes through the first deconvolution layer to obtain the first intermediate training ECG data, the first intermediate training ECG data and the first dictionary training ECG data are concatenated to obtain the second intermediate training ECG data, the second intermediate training ECG data is input into the second convolution layer to obtain the third reconstructed twelve-lead ECG data , when the third reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality training data are consistent, it is confirmed that the decoder in the network architecture of the discrete variational autoencoder is an ECG signal decoder.
[0202] Figure 7A schematic diagram showing a network architecture of an electrocardiogram monitoring model according to an embodiment of the present invention is shown.
[0203] like Figure 7 As shown in the figure, the network architecture of the ECG monitoring model is shown. The ECG monitoring model includes an ECG signal encoder and an ECG signal decoder. The ECG signal encoder includes a three-dimensional spatial encoder and a time series encoder. i Input the three-dimensional space encoder for encoding to obtain the first spatial feature h i , the first spatial feature is input into the first convolutional layer for downsampling, and the first convolutional scale temporal feature h is obtained i (b) , the first convolutional scale temporal feature is input into the second convolutional layer for downsampling, and the second convolutional scale temporal feature h is obtained. i (t) , the first convolutional scale temporal feature and the second convolutional scale temporal feature are input into the self-attention layer for enhancement processing to obtain the first scale temporal feature C i (b) and the second scale temporal feature C i (t) , the second scale time series feature is input into the ECG signal decoder for deconvolution processing to obtain the first intermediate scale time series feature, the first scale time series feature is input into the ECG signal decoder, and is concatenated with the first intermediate scale time series feature to obtain the second intermediate scale time series feature, and the second intermediate scale time series feature is deconvolved to obtain the target twelve-lead ECG image , realizing non-contact twelve-lead ECG monitoring of the monitored object.
[0204] like Figure 8a~Figure 8l As shown, the non-contact ECG monitoring result is the result obtained after non-contact ECG monitoring of the monitored object according to the method of ECG monitoring based on electromagnetic signals of the present invention. Without contacting the monitored object, the electromagnetic signal transceiver front end is used to transmit electromagnetic signals to the target of the monitored object and receive the initial electromagnetic echo signal, and then the received initial electromagnetic echo signal is processed to obtain the non-contact ECG monitoring result. The contact ECG waveform is obtained by sticking the sticky electrode to the target of the monitored object, and then connecting it to the ECG monitor through the ECG wire, so as to monitor the monitored object and obtain the contact ECG waveform. By performing non-contact and contact ECG monitoring on twelve different parts of the monitored object, the following is obtained. Figure 8a~Figure 8l A comparative schematic diagram of non-contact and contact ECG monitoring with twelve leads is shown.
[0205] Figure 8aA schematic diagram showing a comparison between a non-contact first-lead ECG monitoring result and a synchronously monitored contact first-lead ECG waveform according to an embodiment of the present invention is shown.
[0206] like Figure 8a As shown, a comparison is shown between the non-contact first-lead ECG monitoring result and the synchronously monitored contact first-lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage, wherein the first lead can be represented as lead I, the solid line waveform is the non-contact first-lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact first-lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.890. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0207] Figure 8b A schematic diagram showing a comparison between a non-contact second-lead ECG monitoring result and a synchronously monitored contact second-lead ECG waveform according to an embodiment of the present invention is shown.
[0208] like Figure 8b As shown, a comparison is shown between the non-contact second lead ECG monitoring result and the synchronously monitored contact second lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage, wherein the second lead can be represented by lead II, the solid line waveform is the non-contact second lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact second lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.972. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0209] Figure 8c A schematic diagram showing a comparison between a non-contact third-lead ECG monitoring result and a synchronously monitored contact third-lead ECG waveform according to an embodiment of the present invention is shown.
[0210] like Figure 8c As shown, a comparison is shown between the non-contact third-lead ECG monitoring result and the synchronously monitored contact third-lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage, wherein the third lead can be represented as lead III, the solid line waveform is the non-contact third-lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact third-lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.923. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0211] Figure 8dA schematic diagram showing a comparison between a non-contact pressurized unipolar right upper limb lead ECG monitoring result and a synchronously monitored contact pressurized unipolar right upper limb lead ECG waveform according to an embodiment of the present invention.
[0212] like Figure 8d As shown, the non-contact pressurized unipolar right upper limb lead ECG monitoring result is compared with the synchronously monitored contact pressurized unipolar right upper limb lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage. Among them, the pressurized unipolar right upper limb lead can be represented by aVR lead, the solid line waveform is the non-contact pressurized unipolar right upper limb lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact pressurized unipolar right upper limb lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.971. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0213] Figure 8e A schematic diagram showing a comparison between a non-contact pressurized unipolar left upper limb lead ECG monitoring result and a synchronously monitored contact pressurized unipolar left upper limb lead ECG waveform according to an embodiment of the present invention.
[0214] like Figure 8e As shown, the non-contact pressurized unipolar left upper limb lead ECG monitoring result is compared with the synchronously monitored contact pressurized unipolar left upper limb lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage. Among them, the pressurized unipolar left upper limb lead can be represented by aVL lead, the solid line waveform is the non-contact pressurized unipolar left upper limb lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact pressurized unipolar left upper limb lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.821. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0215] Figure 8f A schematic diagram showing a comparison between the non-contact pressurized unipolar left lower limb lead ECG monitoring result and the synchronously monitored contact pressurized unipolar left lower limb lead ECG waveform according to an embodiment of the present invention.
[0216] like Figure 8fAs shown, the non-contact pressurized unipolar left lower limb lead ECG monitoring result is compared with the synchronously monitored contact pressurized unipolar left lower limb lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage. Among them, the pressurized unipolar left lower limb lead can be represented by aVF lead, the solid line waveform is the non-contact pressurized unipolar left lower limb lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact pressurized unipolar left lower limb lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.961. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0217] Figure 8g A schematic diagram showing a comparison between a non-contact first chest lead ECG monitoring result and a synchronously monitored contact first chest lead ECG waveform according to an embodiment of the present invention is shown.
[0218] like Figure 8g As shown, a comparison is shown between the non-contact first chest lead ECG monitoring result and the synchronously monitored contact first chest lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage. The first chest lead can be represented by the V1 lead. The solid line waveform is the non-contact first chest lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact first chest lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.942. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0219] Figure 8h A schematic diagram showing a comparison between a non-contact second chest lead ECG monitoring result and a synchronously monitored contact second chest lead ECG waveform according to an embodiment of the present invention is shown.
[0220] like Figure 8h As shown, a comparison is shown between the non-contact second chest lead ECG monitoring result and the synchronously monitored contact second chest lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage, wherein the second chest lead can be represented by lead V2, the solid line waveform is the non-contact second chest lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact second chest lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.968. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0221] Figure 8i A schematic diagram showing a comparison between a non-contact third chest lead ECG monitoring result and a synchronously monitored contact third chest lead ECG waveform according to an embodiment of the present invention is shown.
[0222] like Figure 8i As shown, a comparison is shown between the non-contact third chest lead ECG monitoring result and the synchronously monitored contact third chest lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage, wherein the third chest lead can be represented by lead V3, the solid line waveform is the non-contact third chest lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact third chest lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.943. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0223] Figure 8j A schematic diagram showing a comparison between a non-contact fourth chest lead ECG monitoring result and a synchronously monitored contact fourth chest lead ECG waveform according to an embodiment of the present invention is shown.
[0224] like Figure 8j As shown, a comparison is shown between the non-contact fourth chest lead ECG monitoring result and the synchronously monitored contact fourth chest lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage. The fourth chest lead can be represented by lead V4. The solid line waveform is the non-contact fourth chest lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact fourth chest lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.970. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0225] Figure 8k A schematic diagram showing a comparison between a non-contact fifth chest lead ECG monitoring result and a synchronously monitored contact fifth chest lead ECG waveform according to an embodiment of the present invention is shown.
[0226] like Figure 8k As shown, a comparison is shown between the non-contact fifth chest lead ECG monitoring result and the synchronously monitored contact fifth chest lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage. The fifth chest lead can be represented by lead V5. The solid line waveform is the non-contact fifth chest lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact fifth chest lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.973. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0227] Figure 8l A schematic diagram showing a comparison between a non-contact sixth chest lead ECG monitoring result and a synchronously monitored contact sixth chest lead ECG waveform according to an embodiment of the present invention is shown.
[0228] like Figure 8lAs shown, a comparison is shown between the non-contact sixth chest lead ECG monitoring result and the synchronously monitored contact sixth chest lead ECG waveform. The horizontal axis can be represented by time, and the vertical axis can be represented by voltage. The sixth chest lead can be represented by lead V6. The solid line waveform is the non-contact sixth chest lead ECG monitoring result obtained after the ECG monitoring model, and the dotted line waveform is the synchronously monitored contact sixth chest lead ECG monitoring result. The correlation coefficient of the two monitoring results is 0.961. It can be seen from the figure that the monitoring results obtained by the non-contact ECG monitoring method of the present invention are highly similar to the contact ECG results.
[0229] like Figure 9a~Figure 9b As shown, the non-contact 12-lead ECG monitoring result is the result obtained after the non-contact ECG monitoring of the monitored object is performed according to the method of the ECG monitoring based on electromagnetic signals of the present invention. Without contacting the monitored object, the electromagnetic signal transceiver front end is used to transmit electromagnetic signals to the target of the monitored object and receive the initial electromagnetic echo signal, and then the received initial electromagnetic echo signal is processed to obtain the non-contact 12-lead ECG monitoring result. The contact measurement result is obtained by sticking the sticky electrode to the target of the monitored object, and then connecting it to the ECG monitor through the ECG wire, so as to monitor the monitored object and obtain the contact measurement result.
[0230] Figure 9a A schematic diagram showing the mean square error distribution of the non-contact twelve-lead electrocardiogram monitoring result compared with the contact measurement result according to an embodiment of the present invention is shown.
[0231] like Figure 9a As shown, the mean square error of the results of non-contact twelve-lead ECG monitoring and contact ECG measurement monitoring of 30 test subjects is compared. The horizontal axis can be represented by the mean square error, and the vertical axis can be represented by the cumulative probability. It can be seen from the figure that the smaller the mean square error, the smaller the error of the monitoring results obtained by the two ECG measurements, and the mean square errors of the results monitored by the non-contact twelve-lead ECG monitoring method of the present invention are all small values.
[0232] Figure 9b A schematic diagram showing waveform correlation between a non-contact twelve-lead electrocardiogram monitoring result and a contact measurement result according to an embodiment of the present invention is shown.
[0233] like Figure 9b As shown, the waveform correlation of the results of non-contact twelve-lead ECG monitoring and contact ECG measurement monitoring of 30 test subjects is compared. The horizontal axis can be represented by the correlation coefficient, and the vertical axis can be represented by the cumulative probability. It can be seen from the figure that the larger the correlation coefficient, the closer the monitoring results obtained by the two ECG measurements are, and the correlation coefficients of the results monitored by the non-contact twelve-lead ECG monitoring method of the present invention are all large values.
[0234] Fig.10 The structure block diagram of the device for electrocardiogram monitoring based on electromagnetic signals according to an embodiment of the present invention is schematically shown.
[0235] like Fig.10 As shown, the device for electrocardiogram monitoring based on electromagnetic signals applied to a receiver in this embodiment includes: a first acquisition module 1010 , a compensation module 1020 , a first processing module 1030 and a second processing module 1040 .
[0236] The first acquisition module 1010 is used to acquire an initial electromagnetic echo signal, wherein the initial electromagnetic echo signal represents a plurality of spatially overlapped signals corresponding to the motion state of the target within a predetermined period of time. The first acquisition module 1010 can be used to perform the operation S210 described above, which will not be described in detail here.
[0237] The compensation module 1020 is used to compensate the initial electromagnetic echo signal according to the signal phase difference corresponding to the initial electromagnetic echo signal to obtain the target motion time series three-dimensional data, wherein the target motion time series three-dimensional data includes the time series space motion information obtained by mapping the time series motion state of the target in the three-dimensional space. The compensation module 1020 can be used to perform the operation S220 described above, which will not be described in detail here.
[0238] The first processing module 1030 is used to process the target motion time series three-dimensional data using the ECG signal encoder of the ECG monitoring model to obtain multi-scale time series features, wherein the multi-scale time series features represent the time series information of the target mechanical motion at least two scales. The first processing module 1030 can be used to perform the operation S230 described above, which will not be described in detail here.
[0239] The second processing module 1040 is used to process the multi-scale time series features using the ECG signal decoder of the ECG monitoring model to obtain a twelve-lead ECG image, wherein the twelve-lead ECG image is used to characterize the change in the motion state of the target within a predetermined time. The second processing module 1040 can be used to perform the operation S240 described above, which will not be repeated here.
[0240] According to an embodiment of the present invention, the compensation module 1030 includes: a first obtaining submodule, a first compensation submodule and a first filtering submodule.
[0241] The first obtaining submodule is used to obtain the signal phase difference based on the position information of the antenna receiving array and the bandwidth of the initial electromagnetic echo signal.
[0242] The first compensation submodule is used to perform phase compensation on the initial electromagnetic echo signal according to the signal phase difference to obtain a target electromagnetic echo signal, wherein the target electromagnetic echo signal includes a plurality of signals that are spatially separated from each other.
[0243] The first filtering submodule is used to filter the target electromagnetic echo signal to obtain the target motion time series three-dimensional data.
[0244] According to an embodiment of the present invention, the first filtering submodule includes: a first splitting unit, a first timing difference unit and a first obtaining unit.
[0245] The first splitting unit is used to split the electromagnetic echo signal corresponding to each spatial point to obtain multi-channel electromagnetic echo data corresponding to each spatial point, wherein the multi-channel electromagnetic echo data includes real channel electromagnetic echo data, imaginary channel electromagnetic echo data and phase channel electromagnetic echo data.
[0246] The first time series difference unit is used to perform time series difference processing on the multi-channel electromagnetic echo data corresponding to each spatial point to obtain the multi-channel motion time series data corresponding to each spatial point.
[0247] The first obtaining unit is used to obtain target motion time series three-dimensional data according to the multi-channel motion time series data corresponding to all spatial points in the three-dimensional space.
[0248] According to an embodiment of the present invention, the first processing module 1030 includes: a first encoding submodule and a second encoding submodule.
[0249] The first encoding submodule is used to input the target motion time series three-dimensional data into the three-dimensional space encoder for encoding to obtain a first space feature, wherein the first space feature is a feature vector of a predetermined dimension.
[0250] The second encoding submodule is used to input the first spatial feature into the time series encoder for processing to obtain a multi-scale time series feature.
[0251] According to an embodiment of the present invention, the second encoding submodule includes: a first downsampling unit, a second downsampling unit and a first enhancement unit.
[0252] The first downsampling unit is used to input the first spatial feature into the first convolution layer for downsampling processing to obtain a first convolution scale temporal feature.
[0253] The second downsampling unit is used to input the first convolutional scale time series feature into the second convolutional layer for downsampling processing to obtain the second convolutional scale time series feature.
[0254] The first enhancement unit is used to input the first convolutional scale time series feature and the second convolutional scale time series feature into the self-attention layer for enhancement processing to obtain multi-scale time series features.
[0255] According to an embodiment of the present invention, the second processing module 1040 includes: a first deconvolution submodule, a first concatenation submodule and a second deconvolution submodule.
[0256] The first deconvolution submodule is used to input the second-scale time series features into the ECG signal decoder for deconvolution processing to obtain the first intermediate-scale time series features.
[0257] The first splicing submodule is used to input the first scale time series feature into the ECG signal decoder, and splice it with the first intermediate scale time series feature to obtain the second intermediate scale time series feature, wherein the dimension of the first scale time series feature is consistent with the dimension of the first intermediate scale time series feature.
[0258] The second deconvolution submodule is used to perform deconvolution processing on the second intermediate-scale time series features to obtain a twelve-lead electrocardiogram image.
[0259] According to an embodiment of the present invention, any multiple modules among the first acquisition module 1010, the compensation module 1020, the first processing module 1030, and the second processing module 1040 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first acquisition module 1010, the compensation module 1020, the first processing module 1030, and the second processing module 1040 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the first acquisition module 1010, the compensation module 1020, the first processing module 1030 and the second processing module 1040 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.
[0260] Fig.11 The structural block diagram of the training device of the electrocardiogram monitoring model according to an embodiment of the present invention is schematically shown.
[0261] like Fig.11As shown, the device for electrocardiogram monitoring based on electromagnetic signals applied to a receiver of this embodiment includes: a second acquisition module 1110 , an encoding module 1120 , a decoding module 1130 , a loss module 1140 and a training module 1150 .
[0262] The second acquisition module 1110 is used to acquire a first sample data set, wherein the first sample data set includes training time series three-dimensional data and twelve-lead ECG single modality label data. The first acquisition module 1110 can be used to perform the operation S310 described above, which will not be described in detail here.
[0263] The encoding module 1120 is used to input the training time series three-dimensional data into the ECG signal encoder to be trained for processing to obtain a first multi-scale sample time series feature, wherein the first multi-scale sample time series feature represents the time series information of the target mechanical motion at least two scales. The encoding module 1120 can be used to perform the operation S320 described above, which will not be repeated here.
[0264] The decoding module 1130 is used to input the first multi-scale training time series feature into the ECG signal decoder to be trained for processing to obtain the first reconstructed 12-lead ECG data. The decoding module 1130 can be used to perform the operation S330 described above, which will not be described in detail here.
[0265] The loss module 1140 is used to obtain a first loss value based on the loss function according to the first reconstructed 12-lead ECG data and the 12-lead ECG single-modality training data. The loss module 1140 can be used to perform the operation S340 described above, which will not be described in detail here.
[0266] The training module 1150 is used to obtain a trained ECG monitoring model by adjusting the model parameters of the ECG monitoring model according to the first loss value. The training module 1150 can be used to perform the operation S350 described above, which will not be described in detail here.
[0267] According to an embodiment of the present invention, the training module 1150 includes: a first adjustment submodule, a first output submodule, a second obtaining submodule and a second adjustment submodule.
[0268] The first adjustment submodule is used to fix the model parameters of the ECG signal encoder of the ECG monitoring model and adjust the model parameters of the ECG signal decoder of the ECG monitoring model according to the first loss value to obtain an intermediate model.
[0269] The first output submodule is used to input the first sample data set into the intermediate model and output the second reconstructed twelve-lead ECG data.
[0270] The second obtaining submodule is used to obtain a second loss value based on the loss function and the second reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality label data.
[0271] The second adjustment submodule is used to fix the model parameters of the ECG signal decoder of the ECG monitoring model and adjust the model parameters of the ECG signal encoder of the ECG monitoring model based on the second loss value to obtain a trained ECG monitoring model.
[0272] According to an embodiment of the present invention, any multiple modules of the second acquisition module 1110, the encoding module 1120, the decoding module 1130, the loss module 1140 and the training module 1150 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the second acquisition module 1110, the encoding module 1120, the decoding module 1130, the loss module 1140 and the training module 1150 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the second acquisition module 1110, the encoding module 1120, the decoding module 1130, the loss module 1140 and the training module 1150 may be at least partially implemented as a computer program module, which may perform a corresponding function when executed.
[0273] Fig.12 The block diagram schematically shows an electronic device for an electrocardiogram monitoring method based on electromagnetic signals according to an embodiment of the present invention.
[0274] like Fig.12As shown, the electronic device according to an embodiment of the present invention includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 to a random access memory (RAM) 1203. The processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include an onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.
[0275] In RAM 1203, various programs and data required for the operation of the electronic device are stored. The processor 1201, ROM 1202, and RAM 1203 are connected to each other via a bus 1204. The processor 1201 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 1202 and / or RAM 1203. It should be noted that the program can also be stored in one or more memories other than ROM 1202 and RAM 1203. The processor 1201 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in one or more memories.
[0276] According to an embodiment of the present invention, the electronic device may further include an input / output (I / O) interface 1205, which is also connected to the bus 1204. The electronic device 1200 may further include one or more of the following components connected to the I / O interface 1205: an input portion 1206 including a keyboard, a mouse, etc.; an output portion 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1208 including a hard disk, etc.; and a communication portion 1209 including a network interface card such as a LAN card, a modem, etc. The communication portion 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage portion 1208 as needed.
[0277] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.
[0278] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1202 and / or RAM 1203 described above and / or one or more memories other than ROM 1202 and RAM 1203.
[0279] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method for electrocardiogram monitoring based on electromagnetic signals provided by the embodiment of the present invention.
[0280] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 1201. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0281] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1209, and / or installed from the removable medium 1211. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0282] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.
[0283] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention may be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All of these combinations and / or combinations fall within the scope of the present invention.
Claims
1. A method for electrocardiogram monitoring based on electromagnetic signals, characterized in that: include: Acquire an initial electromagnetic echo signal, wherein the initial electromagnetic echo signal represents a plurality of spatially overlapped signals corresponding to a motion state of the target within a predetermined period of time; The initial electromagnetic echo signal is compensated according to the signal phase difference corresponding to the initial electromagnetic echo signal to obtain target motion time series three-dimensional data, wherein the target motion time series three-dimensional data includes time series spatial motion information obtained by mapping the time series motion state of the target in the three-dimensional space; The target motion time series three-dimensional data is processed by using an ECG signal encoder of an ECG monitoring model to obtain a multi-scale time series feature, wherein the multi-scale time series feature represents the time series information of the target mechanical motion at least two scales, and the ECG signal encoder includes a three-dimensional space encoder and a time series encoder; the time series encoder includes a first convolution layer, a second convolution layer and a self-attention layer; the target motion time series three-dimensional data is processed by using the ECG signal encoder of the ECG monitoring model to obtain a multi-scale time series feature, including: inputting the target motion time series three-dimensional data into the three-dimensional space encoder for encoding to obtain a first spatial feature, wherein the first spatial feature is a feature vector of a predetermined dimension; inputting the first spatial feature into the first convolution layer for downsampling processing to obtain a first convolution scale time series feature; inputting the first convolution scale time series feature into the second convolution layer for downsampling processing to obtain a second convolution scale time series feature; inputting the first convolution scale time series feature and the second convolution scale time series feature into the self-attention layer for enhancement processing to obtain the multi-scale time series feature; The multi-scale time series features are processed using an ECG signal decoder of an ECG monitoring model to obtain a twelve-lead ECG image, wherein the twelve-lead ECG image is used to characterize the change in the motion state of the target within a predetermined time.
2. The method according to claim 1, characterized in that: The method of compensating the initial electromagnetic echo signal according to the signal phase difference corresponding to the initial electromagnetic echo signal to obtain the target motion time series three-dimensional data includes: Obtaining the signal phase difference based on the position information of the antenna receiving array and the bandwidth of the initial electromagnetic echo signal; According to the signal phase difference, phase compensation is performed on the initial electromagnetic echo signal to obtain a target electromagnetic echo signal, wherein the target electromagnetic echo signal includes a plurality of signals that are spatially separated from each other; The target electromagnetic echo signal is filtered to obtain the target motion time series three-dimensional data.
3. The method according to claim 2, characterized in that The target electromagnetic echo signal includes an electromagnetic echo signal corresponding to each spatial point in the three-dimensional space; The filtering process of the target electromagnetic echo signal to obtain the target motion time series three-dimensional data includes: Splitting the electromagnetic echo signal corresponding to each spatial point to obtain multi-channel electromagnetic echo data corresponding to each spatial point, wherein the multi-channel electromagnetic echo data includes real channel electromagnetic echo data, imaginary channel electromagnetic echo data and phase channel electromagnetic echo data; Performing time series difference processing on the multi-channel electromagnetic echo data corresponding to each spatial point to obtain multi-channel motion time series data corresponding to each spatial point; The target motion time series three-dimensional data is obtained according to the multi-channel motion time series data corresponding to all spatial points in the three-dimensional space.
4. The method according to claim 1, characterized in that: The multi-scale time series features include a first-scale time series feature and a second-scale time series feature; The electrocardiogram signal decoder using the electrocardiogram monitoring model processes the multi-scale time series features to obtain a twelve-lead electrocardiogram image, including: Inputting the second-scale time series features into the ECG signal decoder for deconvolution processing to obtain first intermediate-scale time series features; Inputting the first-scale time series feature into the ECG signal decoder, and concatenating it with the first intermediate-scale time series feature to obtain a second intermediate-scale time series feature, wherein the dimension of the first-scale time series feature is consistent with the dimension of the first intermediate-scale time series feature; The second intermediate-scale time series features are subjected to inverse convolution processing to obtain the twelve-lead electrocardiogram image.
5. A training method for an electrocardiogram monitoring model, characterized in that: The ECG monitoring model includes an ECG signal encoder and an ECG signal decoder, including: Acquire a first sample data set, wherein the first sample data set includes training time-series three-dimensional data and twelve-lead electrocardiogram single-modality label data; Inputting the training time-series three-dimensional data into the ECG signal encoder to be trained for processing to obtain a first multi-scale sample time-series feature, wherein the first multi-scale sample time-series feature represents the time-series information of the target mechanical motion at least two scales, the ECG signal encoder to be trained includes a three-dimensional space encoder to be trained and a time-series encoder to be trained; the time-series encoder to be trained includes a first convolutional layer, a second convolutional layer and a self-attention layer; inputting the training time-series three-dimensional data into the ECG signal encoder to be trained for processing to obtain a first multi-scale sample time-series feature, including: inputting the training time-series three-dimensional data into the ECG signal encoder to be trained The first training spatial feature is encoded by the trained three-dimensional spatial encoder to obtain a first training spatial feature, wherein the first training spatial feature is a feature vector of a predetermined dimension; the first training spatial feature is input into the first convolution layer of the time encoder to be trained for downsampling processing to obtain a first training convolution scale time series feature; the first training convolution scale time series feature is input into the second convolution layer for downsampling processing to obtain a second training convolution scale time series feature; the first training convolution scale time series feature and the second training convolution scale time series feature are input into the self-attention layer for enhancement processing to obtain the first multi-scale sample time series feature; Inputting the first multi-scale sample time series feature into the ECG signal decoder to be trained for processing to obtain first reconstructed twelve-lead ECG data; Based on the loss function, obtaining a first loss value according to the first reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality training data; According to the first loss value, a trained ECG monitoring model is obtained by adjusting the model parameters of the ECG monitoring model.
6. The method according to claim 5, characterized in that The step of obtaining a trained ECG monitoring model by adjusting the model parameters of the ECG monitoring model according to the first loss value includes: According to the first loss value, fixing the model parameters of the ECG signal encoder of the ECG monitoring model, adjusting the model parameters of the ECG signal decoder of the ECG monitoring model, and obtaining an intermediate model; Inputting the first sample data set into the intermediate model, and outputting second reconstructed twelve-lead ECG data; Based on the loss function, obtaining a second loss value according to the second reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality label data; Based on the second loss value, the model parameters of the ECG signal decoder of the ECG monitoring model are fixed, and the model parameters of the ECG signal encoder of the ECG monitoring model are adjusted to obtain the trained ECG monitoring model.
7. An electrocardiogram monitoring device based on electromagnetic signals, characterized in that: include: A first acquisition module is used to acquire an initial electromagnetic echo signal, wherein the initial electromagnetic echo signal represents a plurality of spatially overlapped signals corresponding to a motion state of a target within a predetermined period of time; A compensation module, used for compensating the initial electromagnetic echo signal according to a signal phase difference corresponding to the initial electromagnetic echo signal, to obtain target motion time series three-dimensional data, wherein the target motion time series three-dimensional data includes time series space motion information obtained by mapping the time series motion state of the target in the three-dimensional space; A first processing module is used to process the target motion time series three-dimensional data using an ECG signal encoder of an ECG monitoring model to obtain a multi-scale time series feature, wherein the multi-scale time series feature represents the time series information of the target mechanical motion at least two scales, and the ECG signal encoder includes a three-dimensional space encoder and a time series encoder; the time series encoder includes a first convolution layer, a second convolution layer and a self-attention layer; using the ECG signal encoder of the ECG monitoring model to process the target motion time series three-dimensional data to obtain a multi-scale time series feature, including: inputting the target motion time series three-dimensional data into the three-dimensional space encoder for encoding to obtain a first spatial feature, wherein the first spatial feature is a feature vector of a predetermined dimension; inputting the first spatial feature into the first convolution layer for downsampling processing to obtain a first convolution scale time series feature; inputting the first convolution scale time series feature into the second convolution layer for downsampling processing to obtain a second convolution scale time series feature; inputting the first convolution scale time series feature and the second convolution scale time series feature into the self-attention layer for enhancement processing to obtain the multi-scale time series feature; The second processing module is used to process the multi-scale time series features using the ECG signal decoder of the ECG monitoring model to obtain a twelve-lead ECG image, wherein the twelve-lead ECG image is used to characterize the change in the motion state of the target within a predetermined time.
8. A training device for an electrocardiogram monitoring model, characterized in that: The ECG monitoring model includes an ECG signal encoder and an ECG signal decoder, including: A second acquisition module is used to acquire a first sample data set, wherein the first sample data set includes training time series three-dimensional data and twelve-lead ECG single-modality label data; The encoding module is used to input the training time series three-dimensional data into the ECG signal encoder to be trained for processing to obtain a first multi-scale sample time series feature, wherein the first multi-scale sample time series feature represents the time series information of the target mechanical motion at least two scales, the ECG signal encoder to be trained includes a three-dimensional space encoder to be trained and a time series encoder to be trained; the time series encoder to be trained includes a first convolution layer, a second convolution layer and a self-attention layer; the training time series three-dimensional data is input into the ECG signal encoder to be trained for processing to obtain the first multi-scale sample time series feature, including: inputting the training time series three-dimensional data into the ECG signal encoder to be trained for processing to obtain the first multi-scale sample time series feature The three-dimensional spatial encoder to be trained is encoded to obtain a first training spatial feature, wherein the first training spatial feature is a feature vector of a predetermined dimension; the first training spatial feature is input into the first convolution layer of the temporal encoder to be trained for downsampling processing to obtain a first training convolution scale temporal feature; the first training convolution scale temporal feature is input into the second convolution layer for downsampling processing to obtain a second training convolution scale temporal feature; the first training convolution scale temporal feature and the second training convolution scale temporal feature are input into the self-attention layer for enhancement processing to obtain the first multi-scale sample temporal feature; A decoding module, used for inputting the first multi-scale sample time series feature into an ECG signal decoder to be trained for processing, so as to obtain first reconstructed twelve-lead ECG data; A loss module, configured to obtain a first loss value based on a loss function and according to the first reconstructed twelve-lead ECG data and the twelve-lead ECG single-modality training data; A training module is used to obtain a trained ECG monitoring model by adjusting the model parameters of the ECG monitoring model according to the first loss value.
