Electrocardiogram reconstruction model training method and electrocardiogram reconstruction method and system
The ECG reconstruction model is trained through the flow model, and the information entropy difference interpolation technology is used to solve the problems of inconvenient operation and poor fidelity in the ECG reconstruction method, and high-quality ECG signal reconstruction is achieved, which is suitable for paste-free acquisition.
Patent Information
- Application Number
- CN202310645318.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-09
- Filing Date
- 2023-06-02
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-06-02
AI Technical Summary
The existing electrocardiogram reconstruction methods have problems with inconvenience in operation and skin allergies during the acquisition process, and the reconstruction method based on the deep learning model cannot effectively solve the problem of poor fidelity in reconstructing ECG.
The ECG reconstruction model is trained using the flow model. Through the forward inference and reverse inference process, the model parameters are updated using the information entropy difference vector and the interpolation vector, and the ECG signal is reconstructed by combining the photovoltaic pulse wave signal and the cardiac impact map signal to realize the Information entropy differential interpolation improves the quality of reconstruction.
It improves the fidelity of ECG reconstruction and the interpretability of the model, adapts to different reconstruction source channels, and provides a high-quality ECG acquisition method without pasting.
Smart Images

Figure CN118924304A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine learning, specifically, to the field of deep learning in machine learning, and more specifically, to intelligent medical technology using deep learning, namely, a training method for an electrocardiogram reconstruction model, an electrocardiogram reconstruction method, and an electrocardiogram reconstruction system. Background Art
[0002] In the field of smart healthcare, electrocardiogram (ECG) is widely used in daily health monitoring and clinical diagnosis as a physiological indicator. For example, ECG can be used for auxiliary diagnosis such as arrhythmia classification and heart failure monitoring. In the process of collecting ECG, multi-lead electrodes need to be pasted or fixed to the skin of the user (the user is also called the user). The pasting operation of multi-lead electrodes makes it inconvenient to operate the ECG of the user in daily health testing, and the ECG acquisition electrodes such as multi-lead electrodes that collect signals by pasting or fixing them for a long time may cause skin allergies to the user's skin.
[0003] In view of the above-mentioned problems in the acquisition process of electrocardiogram, the user sets a wearable device to collect physiological signals (integrated sensing signals or signal sources) with the characteristics of integrated sensing acquisition, and converts the physiological signals into electrocardiogram signals (ECG) to realize integrated sensing electrocardiogram acquisition. Among them, the method of converting physiological signals into electrocardiogram signals is the electrocardiogram reconstruction method. The electrocardiogram reconstruction method combines the characteristics of reference experience of electrocardiogram for medical auxiliary diagnosis and the characteristics of easy acquisition of physiological signals with the characteristics of integrated sensing acquisition. Physiological signals with the characteristics of integrated sensing acquisition include photoelectric volumetric pulse wave signals (PPG) and ballistocardiogram signals (BCG). Photoelectric volumetric pulse wave signals and ballistocardiogram signals are physiological signals widely used in clinical or wearable devices. Photoelectric volumetric pulse wave signals are obtained by placing a photoelectric measuring device on the skin of the human body and using the absorption and reflection of light to measure the changes in blood volume under the skin, thereby obtaining pulse wave signals. Ballistocardiogram signals are converted into electrical signals by sensors through the tiny vibrations on the skin surface caused by heart beats. The photoelectric volumetric pulse wave signal and the ballistocardiogram signal can both be non-adhesive and user-imperceptible in terms of the collection principle, and based on this, immersive sensing electrocardiogram collection can be achieved.
[0004] Among the ECG reconstruction methods, the existing ECG reconstruction methods are mainly divided into ECG reconstruction methods based on PPG signals and ECG reconstruction methods based on BCG signals according to the type of signal source used for ECG reconstruction. Among these ECG reconstruction methods, according to the main technical means used in the reconstruction methods, they are mainly divided into ECG reconstruction methods based on non-deep learning models and ECG reconstruction methods based on deep learning models. For example, reference 1 with patent publication number US20210315470 discloses a method for PPG to ECG reconstruction based on a UNet model structure. This method is based on the powerful nonlinear fitting ability of deep learning to learn the mapping relationship from PPG to ECG, wherein the model weights are adjusted in the training stage, and the trained model is used to perform PPG to ECG reconstruction reasoning in the application stage. Reference 2, with patent publication number US20220183606, discloses a method for reconstructing PPG to ECG based on DCT (Discrete Cosine Transform) (ECG reconstruction method without deep learning model). This method is based on DCT, a digital signal processing method, and expresses the mapping relationship from PPG to ECG as coefficients in DCT. The coefficients are fitted in the learning stage, and the learned coefficients are used to reconstruct PPG to ECG in the application stage. Reference 3, with patent publication number CN114041801A, discloses a method for generating adversarial networks based on DCGAN. This method uses a network composed of a generator and a discriminator. In the training stage, the generator and the discriminator are trained adversarially to iteratively improve the similarity between the reconstructed ECG and the real ECG. In the application stage, only the generator is used with the BCG signal as input to output the reconstructed ECG signal. Reference 4 of patent publication number CN114548365A discloses an ECG reconstruction method based on an LSTM neural network model. The method uses an imaging photoplethysmography pulse signal (IPPG) as the input of the LSTM network and an ECG signal as the output of the LSTM neural network model to reconstruct the ECG.
[0005] In summary, the ECG reconstruction method based on immersive sensing signal source can effectively alleviate the problems of cumbersome acquisition process and difficult deployment caused by the adhesive acquisition method of ECG. In particular, the ECG reconstruction method based on deep learning model can significantly improve the representation ability of the mapping relationship between source signals and ECG signals with immersive sensing acquisition characteristics, but it still cannot solve the problem of poor fidelity of reconstructed ECG. Summary of the invention
[0006] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a training method for an electrocardiogram reconstruction model, an electrocardiogram reconstruction method and an electrocardiogram reconstruction system.
[0007] The objective of the present invention is achieved through the following technical solutions:
[0008] According to a first aspect of the present invention, a method for training an electrocardiogram reconstruction model is provided, the method comprising:
[0009] S1. Obtain a training data set consisting of multiple training data pairs, wherein each training data pair includes ECG signal conversion data and immersive perception signal data of the user at the same time, and the ECG signal conversion data is obtained by converting the number of channels of the ECG signal data collected by the user according to the number of channels of the immersive perception signal data, so that the number of channels of the ECG signal conversion data is the number of channels of the immersive perception signal data plus 1; S2. Obtain an electrocardiogram reconstruction model, which is a flow model; S3. Use the training data set to train the electrocardiogram reconstruction model, wherein the training includes the process of forward reasoning and reverse reasoning. During forward reasoning, the electrocardiogram reconstruction model is used to generate synthetic immersive perception signal data and an information entropy difference vector according to the ECG signal conversion data, and the information entropy difference vector indicates the information entropy difference between the ECG signal conversion data and the immersive perception signal data. During reverse reasoning, the electrocardiogram reconstruction model is used to generate synthetic ECG signal conversion data according to the immersive perception signal data and an interpolation vector randomly sampled from a preset probability distribution of the information entropy difference information, and the parameters of the electrocardiogram reconstruction model are updated using the information entropy difference loss corresponding to the information entropy difference vector.
[0010] In some embodiments of the present invention, the information entropy difference loss is determined based on the similarity between the probability distribution of the information entropy difference vector of each training data and a preset Gaussian probability distribution; or, the information entropy difference loss is equal to the mean of the information entropy difference vectors obtained for each training data pair.
[0011] In some embodiments of the present invention, during training, the parameters of the electrocardiogram reconstruction model are updated based on the total loss determined by the information entropy difference loss, the reconstruction loss between the synthesized ECG signal conversion data and the ECG signal conversion data, and the supervision loss between the synthesized integrated perception signal data and the integrated perception signal data.
[0012] In some embodiments of the present invention, the immersive sensing signal data includes ballistocardiogram signal data and photoplethysmogram signal data, and the total loss is determined in the following manner:
[0013]
[0014] Among them, L total represents the total loss, ω 1 represents the first hyperparameter, ω 2 represents the second hyperparameter, ω 3 represents the third hyperparameter, represents the reconstruction loss, Represents the synthetic ECG signal conversion data, x ECG Represents ECG signal conversion data, represents the supervision loss, represents the first supervision loss corresponding to the ballistocardiogram signal data, represents the synthesized ballistocardiogram signal data, x BCG represents the ballistocardiogram signal data, represents the second supervision loss corresponding to the photoplethysmography signal data, represents the synthesized photoplethysmography signal data, x PPG represents the photoplethysmographic signal data, and T represents the information entropy difference loss.
[0015] In some embodiments of the present invention, the channel number conversion is performed in the following manner:
[0016] The ECG signal data is subjected to multiple wavelet transforms, and each wavelet transform obtains an approximate component and a detail component; the approximate component and detail component obtained by the last wavelet transform and the detail component of the wavelet transform before the last wavelet transform are interpolated with zero values to obtain ECG signal conversion data.
[0017] In some embodiments of the present invention, the electrocardiogram reconstruction model is provided with at least one recurrent neural network for extracting time domain features and at least one fully connected neural network for extracting space domain features.
[0018] According to a second aspect of the present invention, there is provided an electrocardiogram reconstruction method based on an electrocardiogram reconstruction model, the method comprising the following steps: C1, obtaining historical data of a user, the historical data comprising ECG signal data and immersive perception signal data of the user at multiple times; C2, preprocessing the physiological signal obtained in step C1 according to a preset processing method to obtain training starting data consisting of multiple data pairs, wherein each data pair comprises ECG signal data and immersive perception signal data of the user in at least one heartbeat cycle; C3, adding 1 to the number of channels of the immersive perception signal data in step C2 as the number of conversion channels; C4, performing channel number conversion on the ECG signal data in the data pair obtained in step C2 to obtain ECG signal conversion data, the ECG signal conversion data and immersive perception signal data in a data pair being converted into ECG signal conversion data. Form a training data pair, and a training data set is composed of multiple training data pairs; C5, use the training method as described in the first aspect of the present invention to train the electrocardiogram reconstruction model multiple times to obtain a trained electrocardiogram reconstruction model; C6, obtain the user's current immersive perception signal, pre-process the user's current immersive perception signal according to a preset processing method, obtain the user's current immersive perception signal data, and sample and generate an insertion vector from the preset information entropy difference information probability distribution between the immersive perception signal and the ECG signal; C7, input the user's current immersive perception signal data and the sampled insertion vector into the trained electrocardiogram reconstruction model to obtain the user's current reconstructed ECG signal conversion data; C8, perform an inverse channel number transformation on the user's current reconstructed ECG signal conversion data to obtain the reconstructed ECG signal data.
[0019] In some embodiments of the present invention, the preset processing method includes one or more of the following methods: filtering, alignment, segmentation, and normalization.
[0020] According to a third aspect of the present invention, there is provided an electrocardiogram reconstruction system, the system comprising:
[0021] An information collection terminal configured to collect physiological signals of a user, and to denoise and output the collected physiological signals; wherein the physiological signals include ECG signals and immersive sensing signals; a data management module configured to receive data output by the information collection terminal and process the data according to a preset data structure; a database configured to store the data processed by the data management module; a reconstruction module configured to execute the method described in the second aspect of the present invention to obtain reconstructed ECG signal data.
[0022] In some embodiments of the present invention, the reconstruction module includes:
[0023] A data reading unit, configured to: acquire and output a training physiological signal of a detection target, wherein the training physiological signal is an ECG signal and an integrated sensing signal of the detection target in a plurality of consecutive heartbeat cycles; a data preprocessing unit, configured to: receive the physiological signal output by the data reading unit, perform preprocessing according to a preset processing method, obtain and output training starting data consisting of a data pair of the detection target in each heartbeat cycle, wherein each data pair is the ECG signal data and the integrated sensing signal data of the detection target in the same heartbeat cycle; a signal type judgment unit, configured to: use the number of channels of the integrated sensing signal data plus 1 as the number of conversion channels, and convert the ECG signal data into ECG signal conversion data, wherein the number of channels of the ECG signal conversion data is equal to the number of conversion channels; a model building unit, configured to: store a pre-initialized electrocardiogram reconstruction model; a training judgment unit, configured to: Determine whether the electrocardiogram reconstruction model stored in the model building unit needs to be trained, and output the training judgment result; the model training unit is configured to: when the training judgment result output by the training judgment unit is that training is needed, execute the training method as described in the first aspect of the present invention to update the parameters of the model in the model building unit; the insertion vector sampling module is configured to: sample and generate an insertion vector from the information entropy difference information probability distribution preset between the integrated perception signal and the ECG signal; the reconstruction reasoning unit is configured to: when the training judgment unit outputs a result that training is not needed, use the electrocardiogram reconstruction model stored in the model building unit as the trained electrocardiogram reconstruction model, and obtain the integrated perception signal in the physiological signal of the detection target and process it according to a preset processing method to obtain the integrated perception signal data and obtain the insertion vector obtained by the insertion vector sampling module, and use the trained electrocardiogram reconstruction model to reconstruct the electrocardiogram.
[0024] Compared with the prior art, the advantages of the present invention are: in the present invention, the channel number conversion is to satisfy the requirement that the ECG reconstruction method proposed by the present invention can adapt to different numbers of reconstruction source channels, wherein the channel number of the ECG signal conversion data is the channel number of the integrated perception signal data plus 1, that is, the channel number of the ECG signal conversion data is one more channel than the channel number of the integrated perception signal data, so that the electrocardiogram reconstruction model can process the information entropy difference information vector introduced and randomly sampled from the information entropy difference information probability distribution function, and utilize the reversibility of the flow model and introduce the information entropy difference between the integrated perception signal source and the ECG signal to train the electrocardiogram reconstruction model, thereby supporting the interpretability of the ECG reconstruction method in terms of modeling principles and improving the quality of the reconstructed ECG. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:
[0026] Figure 1 A schematic diagram of an electrocardiogram reconstruction system provided according to an embodiment of the present invention;
[0027] Figure 2 A schematic diagram of the structure and data flow of an electrocardiogram reconstruction model using a Flow model architecture provided according to an embodiment of the present invention;
[0028] Figure 3 A schematic diagram of the structure of a transformation sub-network provided according to an embodiment of the present invention;
[0029] Figure 4 A schematic diagram of a flow chart of a training method for an electrocardiogram reconstruction model provided according to an embodiment of the present invention;
[0030] Figure 5 A schematic flow chart of an electrocardiogram reconstruction method provided according to an embodiment of the present invention;
[0031] Figure 6 A schematic diagram of a specific implementation flow of an electrocardiogram reconstruction method provided according to an embodiment of the present invention;
[0032] Figure 7 A schematic diagram of an electrocardiogram reconstruction system provided according to an embodiment of the present invention combined with a specific application scenario;
[0033] Figure 8 It is a schematic diagram comparing an original ECG signal and a reconstructed ECG signal provided according to an embodiment of the present invention. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail by specific embodiments below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0035] As mentioned in the background technology, the ECG reconstruction method based on immersive sensing signal source can effectively alleviate the problems of cumbersome acquisition process and difficult deployment caused by the adhesive acquisition method of ECG. In particular, the ECG reconstruction method based on deep learning model can significantly improve the representation ability of the mapping relationship between source signals and ECG signal reconstruction with immersive sensing acquisition characteristics, but it still cannot solve the problem of poor fidelity of reconstructed ECG.
[0036] In order to solve the problem of poor fidelity of reconstructed ECG, the inventor of this application has further analyzed the prior art. The following will take the technologies provided in the four references (reference 1, reference 2, reference 3, reference 4) mentioned in the background technology as an example to explain the analysis process.
[0037] The ECG reconstruction method based on non-deep learning model (such as reference 2 mentioned above) mainly fits the mapping relationship between ECG signal and PPG signal based on a small number of adjustable parameters, and then reconstructs ECG signal based on a defined calculation formula. The nonlinear representation ability of this method is relatively weak, and the reconstructed ECG signal may deviate greatly from the real ECG signal. In this regard, the ECG reconstruction method based on deep learning model (such as reference 1, reference 2, and reference 3 mentioned above) can solve the above-mentioned large deviation problem very well. Although the ECG reconstruction method based on deep learning model has significantly improved the representation ability of the mapping relationship between the source signal with immersive perception acquisition characteristics and the ECG signal reconstruction (thereby solving the problem of large deviation), the ECG reconstruction method based on deep learning model ignores the difference in the information entropy relationship between ECG signal (ECG signal can more faithfully reflect cardiac activity) and source signal, and forces the two to fit (i.e., mapping the source signal with immersive perception acquisition characteristics to ECG signal), which makes the ECG reconstruction method lack of interpretability in modeling principle, and thus cannot solve the problem of poor fidelity of reconstructed ECG. In other words, the existing ECG reconstruction method cannot interpolate the difference in information entropy between the ECG signal and the immersive perception source signal. From the perspective of the perception principle of the reconstructed source signal and the target signal, the modeling principle of the existing ECG reconstruction method lacks interpretability and cannot solve the problem of poor fidelity of the reconstructed ECG. Therefore, in order to solve the above problems, an embodiment of the present invention provides a training method for an electrocardiogram reconstruction model (the electrocardiogram reconstruction model adopts a generative Flow flow model), as well as an ECG reconstruction method and system. Among them, the training method of the electrocardiogram reconstruction model utilizes the reversibility of the flow model and introduces the information entropy difference between the immersive perception signal source and the ECG signal to train the electrocardiogram reconstruction model, so as to realize the interpolation of the information entropy difference between the reconstructed source signal (immersive perception signal source) and the target signal (ECG signal), thereby supporting the interpretability of the ECG reconstruction method in the modeling principle, improving the learning ability of the mapping relationship between the reconstructed source signal and the target signal, and then improving the quality of the reconstructed ECG. First, the reversibility of the downstream model is explained. The reversibility of the flow model includes forward reasoning and reverse reasoning. In forward reasoning, the ECG signal is used as input and the cascaded basic building blocks are used for forward calculation. In reverse reasoning, the immersive sensory signal source is used as input and the cascaded basic building blocks are used for reverse calculation. The following is a schematic explanation of the reversibility of the flow model using a basic building block.
[0038] An example of the forward reasoning calculation process can be expressed as:
[0039] ECG 1 , ECG 2 =split(ECG)
[0040] (BCG / PPG)′ 1 =ECG 1 +F(ECG 2 )
[0041] (BCG / PPG)′ 2 =ECG 2 exp(H((BCG / PPG)′ 1 ))+G((BCG / PPG)′ 1 )
[0042] (BCG / PPG)′=Concat((BCG / PPG)′ 1 , (BCG / PPG)′ 2 )
[0043] Among them, split(*) means to divide the data into two equal parts according to the time dimension of the signal data, Concat(*) means to merge the data according to the time dimension of the signal data, exp(*) means the exponential function, ECG 1 , ECG 2 Indicates that the ECG signal data is divided into two equal parts: the first half data and the second half data; (BCG / PPG)′ 1 Represents the first equal-division data after forward reasoning calculation, (BCG / PPG)′ 2 represents the second equally divided data after forward reasoning calculation, (BCG / PPG)′ represents that the final output of forward reasoning is the synthesized integrated perception signal data. F(*) represents the first transformation subnetwork, H(*) represents the second transformation subnetwork, and G(*) represents the third transformation subnetwork.
[0044] An example of the reverse reasoning calculation process can be expressed as:
[0045] (BCG / PPG) 1 , (BCG / PPG) 2 =split((BCG / PPG))
[0046] ECG 2 =((BCG / PPG) 2 -G((BCG / PPG) 1 ))·exp(-H((BCG / PPG) 1 )),
[0047] ECG 1 =(BCG / PPG) 1 -F(ECG′ 2 )
[0048] ECG′=Concat(ECG′ 1 , ECG′ 2 )
[0049] Among them, (BCG / PPG) 1 , (BCG / PPG) 2 Indicates that the integrated sensing signal data (BCG / PPG) is divided into two equal parts: the first and second equal parts, ECG′ 1 Represents the first half of the data after reverse reasoning calculation, ECG′ 2 Represents the second equally divided data after reverse reasoning calculation, and ECG′ indicates that the final output of reverse reasoning is the reconstructed ECG signal data. It should be noted that the transformation subnetwork in the basic building block can be composed of the first transformation subnetwork and the second transformation subnetwork, or can be composed of the first transformation subnetwork, the second transformation subnetwork, and the third transformation subnetwork. Here, the first transformation subnetwork, the second transformation subnetwork, and the third transformation subnetwork are selected to form the basic building block for the convenience of description to explain the reversibility. When the basic building block composed of the first transformation subnetwork and the second transformation subnetwork is used to illustrate the reversibility, the corresponding items of G(*) in the above process diagram can be omitted. For example, the example of the forward reasoning calculation process omits the corresponding items of G(*) and is expressed as: (BCG / PPG)′ 2 =ECG 2 exp(H((BCG / PPG)′ 1 )).
[0050] It should be noted that, in the embodiment of the present invention, the reversibility of the flow model is introduced when reconstructing the ECG signal. At the same time, the real integrated perception signal and the randomly sampled difference signal distribution are mapped to the ECG signal during the reverse reasoning of the flow model. In other words, the probability distribution of the information entropy difference information between the reconstructed source signal and the ECG signal is preset and randomly sampled, and the sampled vector is obtained. In other words, before the reverse reasoning starts, the probability distribution of the information entropy difference information between the reconstructed source signal (integrated perception signal) and the ECG signal is preset and randomly sampled, and the sampled vector is combined with the integrated perception signal and input into the electrocardiogram reconstruction model to achieve the information entropy difference interpolation between the reconstructed source signal and the target signal (ECG signal), thereby improving the fidelity of the reconstructed ECG and ensuring interpretability in the modeling process. Among them, when training the ECG reconstruction model, the reversibility of the flow model is used to reversely map the immersive perception signal source and the information entropy difference signal distribution vector to the ECG signal; the immersive perception signal corresponding to the ECG signal is output forwardly, realizing the bidirectional mapping between the ECG signal and the immersive perception signal and ensuring that the information entropy difference between the immersive perception signal source and the ECG signal is interpolated during the modeling process, so that the modeling process of the ECG reconstruction method satisfies the physiological relationship between the reconstructed source signal and the target signal (that is, it satisfies the interpretability of the ECG reconstruction method in the modeling principle), significantly improving the quality of the reconstructed ECG. Continuing to explain the above reverse reasoning calculation process example, before the reverse reasoning begins, the information entropy difference information probability distribution between the reconstructed source signal (immersive perception signal) and the ECG signal is randomly sampled, and the interpolation vector obtained by sampling is combined with the immersive perception signal data (BCG / PPG) and input into the flow model to realize the information entropy difference interpolation between the reconstructed source signal and the ECG signal, improving the fidelity of the reconstructed ECG, and also ensuring interpretability at the level of the modeling process. The present invention designs an adaptive method for the number of reconstruction source channels to solve the problem of the non-fixed number of channels of the immersive perception signal source in the ECG signal reconstruction method proposed by the present invention. The number of channels is adjusted according to the number of input signal channels by using channel number conversion (such as wavelet transform level) to support the input of variable channel immersive perception signal source. An ECG reconstruction method and system provided by an embodiment of the present invention reconstructs ECG based on the immersive perception signal and the sampled difference signal distribution and using an electrocardiogram reconstruction model. In this way, an ECG acquisition method using immersive perception signals can be provided when ECG acquisition equipment is unavailable.
[0051] According to one embodiment of the present invention, there is provided an electrocardiogram reconstruction method based on an electrocardiogram reconstruction model, the method comprising the following steps: C1, obtaining historical data of a user, the historical data comprising ECG signal data and immersive perception signal data of the user at multiple times; C2, preprocessing the physiological signal obtained in step C1 according to a preset processing method to obtain training starting data consisting of multiple data pairs, wherein each data pair comprises ECG signal data and immersive perception signal data of the user in at least one heartbeat cycle; C3, adding 1 to the number of channels of the immersive perception signal data in step C2 as the number of conversion channels; C4, performing channel number conversion on the ECG signal data in the data pair obtained in step C2 to obtain ECG signal conversion data, which is composed of one ECG signal conversion data and immersive perception signal data in one data pair. A training data pair, which is composed of a training data set consisting of multiple training data pairs; C5, using the training method of the electrocardiogram reconstruction model provided by the present invention to train the electrocardiogram reconstruction model multiple times to obtain a trained electrocardiogram reconstruction model; C6, obtaining the user's current immersive perception signal, preprocessing the user's current immersive perception signal according to a preset processing method to obtain the user's current immersive perception signal data, and sampling and generating an insertion vector from the preset information entropy difference information probability distribution between the reconstructed source signal (immersive perception signal) and the ECG signal; C7, inputting the user's current immersive perception signal data and the insertion vector generated by sampling into the trained electrocardiogram reconstruction model to obtain the user's current reconstructed ECG signal conversion data; C8, performing an inverse transform of the number of channels on the user's current reconstructed ECG signal conversion data to obtain the reconstructed ECG signal data.
[0052] According to one embodiment of the present invention, Figure 1As shown, an electrocardiogram reconstruction system is provided, the system includes a signal acquisition terminal 400, a data management module 406, a database 407 and an ECG reconstruction module 408. The information acquisition terminal 400 is configured to acquire physiological signals of the user, and to perform denoising processing on the acquired physiological signals and output them; wherein the physiological signals include ECG signals and immersive sensing signals; the data management module 406 is configured to receive the data output by the information acquisition terminal and process them according to a preset data structure; the database 407 is configured to store the data processed by the data management module; the reconstruction module (that is, the ECG reconstruction module 408) is configured to execute an electrocardiogram reconstruction method based on an electrocardiogram reconstruction model to obtain reconstructed ECG signal data.The ECG reconstruction module 408 includes a data reading unit 4081, a data preprocessing unit 4082, a signal type judgment unit 4083, a model building unit 4084, a training judgment unit 4085, a model training unit 4086, an ECG reconstruction reasoning unit 4087, and an insertion vector sampling module 4088; wherein the data reading unit 4081 is configured to: obtain and output the training physiological signal of the detection target, the training physiological signal is the ECG signal and the integrated sensing signal of the detection target in multiple consecutive heartbeat cycles; the data preprocessing unit 4082 is configured to: obtain and output the training physiological signal of the detection target, the training physiological signal is the ECG signal and the integrated sensing signal of the detection target in multiple consecutive heartbeat cycles; The present invention is configured to: receive the physiological signal output by the data reading unit, perform preprocessing according to a preset processing method, obtain and output the training starting data composed of the data pairs of the detection target in each heartbeat cycle, wherein each data pair is the ECG signal data and the integrated sensing signal data of the detection target in the same heartbeat cycle; the signal type judgment unit 4083 is configured to: use the number of channels of the integrated sensing signal data plus 1 as the number of conversion channels, and convert the ECG signal data into ECG signal conversion data, wherein the number of channels of the ECG signal conversion data is equal to the number of conversion channels; the model construction The construction unit 4084 is configured to store the pre-initialized ECG reconstruction model; the training judgment unit 4085 is configured to determine whether the ECG reconstruction model needs to be trained and output the training judgment result; the model training unit 4086 is configured to execute the training method based on the ECG reconstruction model provided by the present invention to obtain the trained ECG reconstruction model and update the parameters of the model in the model construction unit when the training judgment result output by the training judgment unit is that training is required; the insertion vector sampling module 4088 is configured to extract the reconstructed source signal (integrated perception signal) from the reconstructed source signal ) and the ECG signal by sampling from the information probability distribution of the preset information entropy difference between the ECG signal and the ECG signal to generate an insertion vector; an ECG reconstruction reasoning unit 4087, which is configured as: when the output result of the training judgment unit is that no training is required, the ECG reconstruction model stored in the model building unit is used as the trained ECG reconstruction model, and the integrated perception signal in the physiological signal of the detection target is obtained and processed according to a preset processing method to obtain the integrated perception signal data and obtain the insertion vector obtained by the insertion vector sampling module 4088, and the trained ECG reconstruction model is used to reconstruct the ECG.
[0053] According to one embodiment of the present invention, a method for training an electrocardiogram reconstruction model is provided, the method comprising: S1, obtaining a training data set consisting of a plurality of training data pairs, wherein each training data pair comprises ECG signal conversion data and immersive perception signal data of a user at the same time, the ECG signal conversion data being obtained by performing channel number conversion on the ECG signal data collected by the user according to the channel number of the immersive perception signal data, so that the channel number of the ECG signal conversion data is the channel number of the immersive perception signal data plus 1; S2, obtaining an electrocardiogram reconstruction model, which is a flow model; S3, using the training data set to train the The electrocardiogram reconstruction model is described, wherein the training includes the process of forward reasoning and reverse reasoning. During forward reasoning, the electrocardiogram reconstruction model is used to generate synthetic integrated perception signal data and an information entropy difference vector according to ECG signal conversion data, and the information entropy difference vector indicates the information entropy difference between the ECG signal conversion data and the integrated perception signal data. During reverse reasoning, the electrocardiogram reconstruction model is used to generate synthetic ECG signal conversion data according to the integrated perception signal data and an interpolation vector randomly sampled from a preset probability distribution of the information entropy difference information, and the parameters of the electrocardiogram reconstruction model are updated using the information entropy difference loss corresponding to the information entropy difference vector.
[0054] In order to better explain the present invention, the following mainly explains from the perspective of ECG reconstruction, and in combination with the accompanying drawings, respectively from the construction of the database, the ECG reconstruction model, the processing of the reconstructed data, the training of the ECG reconstruction model, and the ECG reconstruction process.
[0055] 1. Database Construction
[0056] The ECG reconstruction method provided in the embodiment of the present invention can realize ECG reconstruction in the case of an integrated sensing signal source with a non-fixed number of channels. In other words, in the implementation of the present invention, the ECG reconstruction method can meet the situation of different numbers of input signal channels, that is, the integrated sensing signal source can be multiple. According to one embodiment of the present invention, the integrated sensing signal includes one or more of a ballistocardiogram signal and a photoelectric volumetric pulse wave signal. In order to facilitate the explanation of the embodiment of the present invention, the construction process of the database is explained by taking the historical data composed of two signal sources, the ballistocardiogram signal and the photoelectric volumetric pulse wave signal, as an example of the integrated sensing signal and the ECG signal required for training the electrocardiogram reconstruction model. The construction process of the database 407 is explained in combination with the signal acquisition terminal 400 and the data management module 406.
[0057] like Figure 1As shown, the signal acquisition terminal 400 includes a data acquisition unit 4001, a data processing unit 4002, and a data transmission unit 4003. The data acquisition unit 4001 in the signal acquisition terminal 400 simultaneously acquires information of the subject (the subject is also called the user) through a physiological signal sensor to obtain an ECG signal at the same time and an original data set of an immersive sensing signal composed of a ballistocardiogram signal and a photoelectric volume pulse wave signal, and the original data set is sent to the data processing unit 4002 for further processing. The data processing unit 4002 in the signal acquisition terminal 400 can configure different filter cutoff frequencies according to the frequency bandwidth range of the original signal in the original data set, and filter the baseline drift and high-frequency noise, and the filtered signal data is sent to the data transmission unit 4003. The data transmission unit 4003 can transmit the digitized signal to the data management module 406 through Wifi, Bluetooth or wired methods.
[0058] like Figure 1 As shown, according to one embodiment of the present invention, the data management module 406 includes a data receiving unit 4061 and a data management unit 4062. The data receiving unit 4061 receives the data transmitted by the data transmission unit 4003 (the data stream of the ECG signal and the immersive perception signal after basic filtering) and performs data aggregation. The data receiving unit 4061 sends the aggregated data to the data management unit 4062 for standardized data storage and management. The data management unit 4062 can write the received data stream into the database 407 according to a certain data format according to different data channels to form data for ECG reconstruction model training, as well as signal data required for reconstructing ECG (immersive perception signal data required for reconstructing ECG) during specific implementation. The data for ECG reconstruction model training is also the original signal data set in the database 407 (the original signal data set is also the user's historical data, and the historical data includes the user's ECG signal data and immersive perception signal data at multiple times). The database 407 also stores the information entropy difference information probability distribution pre-set between the reconstructed source signal (immersive perception signal) and the ECG signal. Specifically, the pre-set information entropy difference information probability distribution can be understood as setting a deviation information distribution function between the immersive perception signal and the ECG signal. According to one embodiment of the present invention, the deviation information distribution function is a Gaussian distribution. It should be noted that the certain data format here can be understood as a format that is compatible with the storage structure of the database 407. This format is a technology known in the art according to the type of database and will not be repeated here.
[0059] 2. ECG Reconstruction Model
[0060] Since the electrocardiogram reconstruction model of the embodiment of the present invention adopts a generative Flow model, the learning ability of the mapping relationship between the integrated perception signal and the ECG signal can be improved. According to one embodiment of the present invention, the electrocardiogram reconstruction model adopts a Flow model. Figure 2 As shown, the flow model includes multiple cascaded flow model basic building blocks (TimeFlew basic blocks), and adjacent flow model basic building blocks are connected by 1*1 convolution layers (1×1Conv); the flow model basic building blocks are configured to generate output data of the flow model basic building blocks based on the data input to them. Take the lth flow model basic building block composed of 3 transformation sub-networks as an example and combine Figure 2 Explain the forward reasoning process and reverse reasoning process of the ECG reconstruction model of the flow model. Figure 2 In the figure, Length represents the data length of the input l-th stream model basic building block, Length / 2 represents the data length after the data of the input l-th stream model basic building block is split, F, H, G represent the structure of the first transformation sub-network, the structure of the second transformation sub-network, and the structure of the third transformation sub-network; ☉ etc. represent the operation relationship, split represents the two-equal-part splitting function, and Concat represents data merging.
[0061] The data flow in the transformation subnetwork of the lth stream model basic building block in forward reasoning is expressed as
[0062]
[0063]
[0064] The data flow in the sub-transformation network of the lth flow model basic building block in reverse reasoning is expressed as
[0065]
[0066]
[0067] in, Represents the first piece of data input to the lth stream model basic building block in forward reasoning, Represents the second segment of input data of the lth stream model building block in forward reasoning, Represents the first data segment output by the lth stream model building block in forward reasoning. Represents the second segment of data output by the lth stream model building block in forward reasoning. Represents the first piece of data input to the lth stream model building block in reverse reasoning. Represents the second segment of data input to the lth stream model building block in reverse reasoning, Represents the first piece of data output by the lth stream model basic building block in reverse reasoning, It represents the second piece of data output by the lth flow model basic building block in reverse reasoning, F(*) represents the first transformation subnetwork, H(*) represents the second transformation subnetwork, and G(*) represents the third transformation subnetwork.
[0068] According to one embodiment of the present invention, at least one recurrent neural network is provided in the electrocardiogram reconstruction model for extracting time domain features and at least one fully connected neural network is provided for extracting space domain features. The structure of the above-mentioned transformation subnetwork may adopt an existing structure, or an improved structure. Preferably, the model structures of the transformation subnetworks F, H, and G are the same, and the recurrent neural network and the fully connected neural network are used as basic operators. The data stream is first transmitted through the recurrent neural network operator to extract the time domain relationship in the data stream. Subsequently, the output of the recurrent neural network operator is used as the input of the fully connected neural network operator to extract the spatial domain mapping relationship between the reconstructed source signal and the target ECG signal, which can further significantly improve the quality of the reconstructed ECG. Figure 3 As shown, it is a schematic diagram of the network structure of the transformation subnetwork, where, in order to conveniently represent the recurrent neural network, LSTM is represented as a computing unit in the recurrent neural network, X0, X1, ...Xn represent the data input into the recurrent neural network, and h0, h1, ...hn represent the data output by the recurrent neural network. Since the specific structure and function of the recurrent neural network and the fully connected neural network are known to those skilled in the art, the structure and function of the recurrent neural network and the fully connected neural network are not described in detail here.
[0069] 3. Processing of Reconstructed Data
[0070] In the process of ECG reconstruction, the data in the database obtained in the first part needs to be further processed to adapt to the electrocardiogram reconstruction model and ECG reconstruction. Figure 1 Continue with the explanation of data processing.
[0071] The data reading unit 4081 in the ECG reconstruction module 408 is configured to obtain and output the training physiological signal of the detection target, which is the ECG signal and the integrated sensing signal of the detection target in multiple consecutive heartbeat cycles, and can convert the signal data (training physiological signal) from a binary format into a data structure (preset data structure) that can be processed by the software, and then sent to the data preprocessing unit 108 for subsequent data preprocessing. The data structure that can be processed by the software here should be understood as conventional processing in computer processing, which is known to those skilled in the art, and the data structure will not be repeated here.
[0072] The data preprocessing unit 4082 in the ECG reconstruction module 408 is configured to receive the physiological signal output by the data reading unit, perform preprocessing according to a preset processing method, obtain and output the training starting data composed of the data pairs of the detection target in each heartbeat cycle, wherein each data pair is the ECG signal data and the integrated sensing signal data of the detection target in the same heartbeat cycle. Specifically, the data preprocessing unit 4082 is mainly used to convert the data (original ECG signal data, PPG signal data and BCG signal data processing) of the ECG reconstruction model training into the format required by the electrocardiogram reconstruction model input, first refer to the correspondence between the original ECG signal, PPG signal and BCG signal waveform and the cardiophysiological index, that is, the peak point of the R wave of the ECG signal, the trough point of the PPG signal and the J wave peak point of the BCG are aligned in the time domain; then the ECG, PPG and BCG signals are divided into heartbeat level (heartbeat cycle) data according to the above alignment points and form data pairs, and each heartbeat level data segment is normalized to obtain the data pairs in the training starting data. It should be noted that when performing ECG reconstruction, reconstructing the source signal (integrated perception signal), PPG signal and BCG signal, the trough points of the PPG signal and the J peak points of the BCG are aligned in the time domain, and the aligned data are divided into heartbeat level (heart cycle) data. Each heartbeat level data segment is normalized and transformed to obtain the data corresponding to the reconstructed source signal during ECG reconstruction.
[0073] IV. Training of ECG reconstruction model
[0074] The following combination Figure 4 Explain the training process of the ECG reconstruction model. When training the ECG reconstruction model, preferably, the data initialized by the data preprocessing unit 4082 is converted into a training data set for training after the number of channels is converted. Here, the training process of the ECG reconstruction model is mainly explained in detail. The configuration functions of other modules of the ECG reconstruction module 408 (model building unit 4084, training judgment unit 4085, model training unit 4086, ECG reconstruction reasoning unit 4087, and insertion vector sampling module 4088) will be described in detail in the ECG reconstruction process below.
[0075] Specifically, the training method of the electrocardiogram reconstruction model includes: obtaining a training data set composed of multiple training data pairs, wherein each training data pair includes ECG signal conversion data and immersive perception signal data of the user at the same time, and the ECG signal conversion data is obtained by converting the number of channels of the ECG signal data collected by the user according to the number of channels of the immersive perception signal data, so that the number of channels of the ECG signal conversion data is the number of channels of the immersive perception signal data plus 1; obtaining an electrocardiogram reconstruction model, which is a flow model; using the training data set to train the electrocardiogram reconstruction model, wherein the training includes the process of forward reasoning and reverse reasoning, during forward reasoning, using the electrocardiogram reconstruction model to generate synthetic immersive perception signal data and an information entropy difference vector according to the ECG signal conversion data, and the information entropy difference vector indicates the information entropy difference between the ECG signal conversion data and the immersive perception signal data, and during reverse reasoning, using the electrocardiogram reconstruction model to generate synthetic ECG signal conversion data according to the immersive perception signal data and an interpolation vector randomly sampled from a preset probability distribution of the information entropy difference information, and using the information entropy difference loss corresponding to the information entropy difference vector to update the parameters of the electrocardiogram reconstruction model.
[0076] like Figure 4 As shown, it should be noted that the real ECG signal data (ECG signal conversion data) in the data set (that is, the training data set) is input into the Flow reconstruction model (corresponding to the electrocardiogram reconstruction model), and the Flow reconstruction model performs forward reasoning to output the synthesized integrated perception signal and the information entropy difference information vector. The information entropy difference information vector output by the forward reasoning is one of the data dimensions calculated and output by the Flow reconstruction model. The information entropy difference information vector output by the forward reasoning will be used to calculate the loss function and to train the model so that the information entropy difference information is mapped to a Gaussian distribution; the multidimensional data obtained by combining the integrated perception signal in the data set and the information entropy difference information vector randomly sampled from the information entropy difference information probability distribution function is input into the Flow reconstruction model, and the Flow reconstruction model performs reverse reasoning to output the reconstructed ECG; the reconstructed ECG, the synthesized integrated perception signal and the information entropy difference information vector are brought into the loss function, the loss is calculated, and the model weights are updated using the optimizer.
[0077] Among them, each training data pair includes ECG signal conversion data and immersive perception signal data of the user at the same time, and the ECG signal conversion data is obtained by converting the channel number of the ECG signal data collected by the user according to the channel number of the immersive perception signal data, so that the channel number of the ECG signal conversion data is the channel number of the immersive perception signal data plus 1. According to one embodiment of the present invention, the channel number conversion is converted by an adaptive method of reconstructing the number of channels of the source. First, the number of channels of the immersive perception signal data is determined, and then the wavelet transform method is used to meet the situation of different signal channel numbers in the process of reconstructing the ECG signal. Specifically, when the number of input channels of the immersive perception signal source is small, a shallow wavelet transform is used for preprocessing the original ECG signal; when the number of input channels of the immersive perception signal source increases, a deeper wavelet transform is used for preprocessing the original ECG signal to ensure that the number of signal channels of the ECG signal after the wavelet transform corresponds to the number of channels of the immersive perception signal source, thereby meeting the situation of different input signal channel numbers. Specifically, the conversion of the number of channels is to satisfy the requirement that the ECG reconstruction method proposed in the present invention can adapt to different numbers of reconstruction source channels, wherein the number of channels of the ECG signal conversion data is the number of channels of the integrated perception signal data plus 1, that is, the number of channels of the ECG signal conversion data is one more than the number of channels of the integrated perception signal data, so that the electrocardiogram reconstruction model can process the introduced information entropy difference information. In the forward reasoning, the data after the channel number conversion is processed by splicing the data and inputting it into the electrocardiogram reconstruction model; in the reverse reasoning, the interpolation vector randomly sampled from the preset probability distribution of the information entropy difference information and the integrated perception signal data are spliced and input into the electrocardiogram reconstruction model for processing (for example Figure 2 C in the middle represents the concatenated data).
[0078] According to one embodiment of the present invention, the channel number conversion is performed in the following manner: the ECG signal data is subjected to multiple wavelet transforms, and each wavelet transform obtains an approximate component and a detail component; the approximate component and detail component obtained by the last wavelet transform and the detail component of the wavelet transform before the last wavelet transform are interpolated with zero values to obtain ECG signal conversion data. It should be noted that when an integrated sensing signal data is selected for ECG reconstruction, the ECG signal data is subjected to a wavelet transform once to obtain an approximate component and a detail component, that is, the ECG signal conversion data is obtained.
[0079] The following takes the example of the integrated sensing signal consisting of the ballistocardiogram signal and the photoplethysmogram signal to explain the channel number conversion process. The current embodiment uses 2 integrated sensing signals, and the adaptive reconstruction source channel number method performs a secondary wavelet transform on the ECG channel. ECG where φ a,b and ψ a,b Respectively represent the approximate function and detail function of the Bb5 wavelet basis function with a scale of a and a translation of b, N represents the length of the ECG signal, the translation of the wavelet transform in this embodiment is 0 (i.e. b=0), and the scales of the transform are 1 and 2 respectively, firstly, the original ECG signal is used as input, and a transform with a transform scale of 1 (i.e. a=1) is performed, and the approximate component and the first-level detail component are output, then the approximate component is used as input, and a transform with a transform scale of 2 (i.e. a=2) is performed, and the second-level approximate component and the second-level detail component are output, and the second-level approximate component, the first-level detail component and the second-level detail component are interpolated with zero values, expanded to ECG equal-length data, and wavelet reconstruction is performed, and finally converted into ECG signal data consisting of three channels of the second-level approximate component, the first-level detail component and the second-level detail component, and the data structure that meets the requirements of the reconstruction model interface format is output. It should be noted that the reconstructed ECG signal data is obtained in the inverse reconstruction process, and the reconstructed ECG signal data after the channel number is merged is obtained by the inverse transform of the wavelet transform. It should be noted that the secondary approximation component corresponds to the PPG signal component data in the ECG signal data, the primary detail component corresponds to the BCG signal component data in the ECG signal data, and the secondary detail component corresponds to the information entropy difference information component data in the BCG signal component data in the ECG signal data. It should be noted that the adaptive method for reconstructing the number of source channels can be configured in the signal type judgment unit 4083, and the signal type judgment unit 4083 is configured to: add 1 to the number of channels of the immersive perception signal data as the number of conversion channels, and convert the ECG signal data into ECG signal conversion data, wherein the number of channels of the ECG signal conversion data is equal to the number of conversion channels; the adaptive method for reconstructing the number of source channels can be configured to run in the data preprocessing unit 4082. In the embodiment of the present invention, preferably, the data preprocessing unit 4082 is configured to run the above-mentioned adaptive method for reconstructing the number of source channels.
[0080] According to one embodiment of the present invention, during training, the parameters of the electrocardiogram reconstruction model are updated based on the total loss determined by the information entropy difference loss, the reconstruction loss between the synthesized ECG signal conversion data and the ECG signal conversion data, and the supervision loss between the synthesized integrated perception signal data and the integrated perception signal data.
[0081] According to an embodiment of the present invention, the immersive sensing signal data includes ballistocardiogram signal data and photoplethysmogram signal data to determine the total loss in the following manner:
[0082]
[0083] Among them, L total represents the total loss, ω 1 represents the first hyperparameter, ω2 represents the second hyperparameter, ω 3 represents the third hyperparameter, represents the reconstruction loss, Represents the synthetic ECG signal conversion data, x ECG Represents ECG signal conversion data, represents the supervision loss, represents the first supervision loss corresponding to the ballistocardiogram signal data, represents the synthesized ballistocardiogram signal data, x BCG represents the ballistocardiogram signal data, represents the second supervision loss corresponding to the photoplethysmography signal data, represents the synthesized photoplethysmography signal data, x PPG represents the photoplethysmographic signal data, and T represents the information entropy difference loss.
[0084] According to one embodiment of the present invention, the information entropy difference loss is determined based on the similarity between the probability distribution of the information entropy difference vector of each training data and the preset Gaussian probability distribution, and the information entropy difference loss is calculated by the following rule:
[0085]
[0086] Among them, CrossEntropy(*) represents the cross entropy function, p(z) is the probability distribution of the information entropy difference vector z, represents the probability distribution of the input ECG signal data, The value of z is The probability of when , the expression of T is first transformed into the variational form of cross entropy Using the push-forward distribution, the expression of T is transformed into q(ECG) represents the probability distribution of the input ECG signal data, and the information entropy difference loss T represents The similarity between the predicted distribution of z and the true distribution of z, E q(ECG) It represents the expected similarity between the probability of Z output by the flow model inference and the actual probability of Z when all ECGs observed under the unknown ECG probability distribution q(ECG) are taken as input.
[0087] According to one embodiment of the present invention, the information entropy difference loss is equal to the mean of the information entropy difference vectors obtained from each training data pair. The information entropy difference loss is calculated by the following rule:
[0088]
[0089] Where M represents the number of training data pairs, k represents the kth training data pair in the training data set, and N represents the length of BCG, PPG, and ECG data in a data pair. An interpolated vector representing the forward inference output of the streaming model.
[0090] Taking the probability distribution of the information entropy difference vector z as a one-dimensional Gaussian distribution as an example (i.e., standard normal distribution), the formula is explained as follows:
[0091]
[0092] We substitute μ=0,σ=1 into the above formula and get:
[0093]
[0094] p(z) is the probability distribution of the information entropy difference vector z, corresponding to f(x) in formula (2), so it is substituted into the information entropy difference loss formula (1):
[0095]
[0096] In the process of calculating the loss function As a constant, it can be ignored. The z in formula (3) is replaced by , and the subsequent loss calculation formula is derived:
[0097]
[0098] when When expressed in discrete form and the number of training data pairs is M,
[0099] When constructing the loss with a data pair corresponding to the data, according to an embodiment of the present invention, the total loss is determined in the following manner:
[0100] Where N represents the length of BCG, PPG, and ECG data in a data pair. An interpolated vector representing the forward inference output of the streaming model.
[0101] 5. ECG Reconstruction Process
[0102] The following combination Figure 5 This paper describes the electrocardiogram reconstruction process from the perspective of method flow. Figure 5As shown, the specific process of the electrocardiogram reconstruction method is as follows: Step 201 is the starting step, the signal acquisition terminal performs data acquisition, data filtering, and transmits the data to the data management module through the data transmission module. In step 202, the data management module receives the data and stores the data in the data set. In step 203, it is determined whether reconstruction model training is required, wherein the indicators used for determination are relative root mean square error (Relative Root Mean Square Error) and Pearson Correlation Coefficient (Pearson Correlation Coefficient), and the determination standard is whether the indicators have reached the test results on the test set under the model convergence state. If the standard is not reached (the training determination result is that training is required), then model training is required, and step 204 is executed; if the model needs to be used, step 211 is executed; in step 204, the system determines whether the data collected by the data management module is sufficient for ECG reconstruction model training, if not sufficient, it returns to step 201 to continue collecting data, and if sufficient (for example, 5 hours of data), it enters step 205 to read the collected data set and perform preprocessing processes such as data filtering, alignment, segmentation and normalization, and perform wavelet transform decomposition on the ECG signal, and finally outputs the input and output format requirements of the reconstruction model based on Flow. Step 206 mainly divides the ECG signal and the immersive perception signal data segment after preprocessing into training set and test set for subsequent reconstruction model training. In step 207, the Flow-based ECG reconstruction model is mainly constructed and initialized, and basic components such as optimizers for model training are constructed. Then in step 208, the system uses the data set and optimizer to adjust the weights of the reconstruction model and perform model training. In step 209, the system will determine whether the reconstruction model has converged, that is, whether the reconstruction model can reconstruct high-quality ECG signals on the test set. If the model has not converged, it will return to step 208 to continue training the reconstruction model. If the model converges, it will enter step 210 to store the model structure and model weight data. If the system selects the model usage mode, it will enter step 211. In step 211, the system reads the immersive perception signal and performs data preprocessing, constructs a data set that meets the input format requirements of the reconstruction model, and then enters step 212 for model reasoning. In step 212, the system reads the model structure and model weight data stored in step 210, and uses the model weight data to update the model reconstruction model to obtain a trained electrocardiogram reconstruction model; the reconstruction model only needs to obtain the randomly sampled difference vector (also called interpolation vector when discrete data) and the integrated perception signal source obtained in step 211 as input, and the trained electrocardiogram reconstruction model can be used to output a high-quality reconstructed ECG.
[0103] The following combination Figure 1This paper describes the ECG reconstruction process from a system perspective. Figure 1As shown: The reconstruction process of the ECG reconstruction system is as follows: the data acquisition unit 4001 of the signal acquisition terminal 400 collects the information of the subject through the physiological signal sensor to obtain the original data of the ECG and the immersive perception signal, and the original data is sent to 4002 for further processing. The data processing unit 4002 in the signal acquisition terminal 400 can configure different filter cutoff frequencies according to the frequency bandwidth range of the original signal, filter out the baseline drift and high-frequency noise, and the filtered signal is sent to the data transmission unit 4003. The data transmission unit 4003 can transmit the digitized signal to the data receiving unit 4061 for data aggregation through Wifi, Bluetooth or wired methods. The data receiving unit 4061 receives the data stream of the ECG signal and the immersive perception signal after basic filtering, and sends it to the data management unit 4062 for standardized data storage and management. The data management unit 4062 can write the received data stream into the database 407 according to a certain data format according to different data channels to form a data set for ECG reconstruction model training. When the amount of data in the database 407 is sufficient (in step 204), for example, 5 hours of data, preferably, the data is sent to the data reading unit 4081 for data reading for subsequent model training. The data reading unit 4081 can read data from the data set storing the original signal and output it to the data preprocessing unit 4082 according to a specific data structure for subsequent data preprocessing. The data preprocessing unit 4082 mainly implements data filtering according to the bandwidth information of the ECG signal and the integrated perception signal, aligns the signals with reference to the correspondence between different signals and cardiac activities, divides the signals into heartbeat-level data segments, and normalizes the signals; after the preprocessing is completed, the signal type judgment unit 4083 is called to run the adaptive method for reconstructing the number of source channels (that is, to perform a channel number conversion operation), wherein, according to the number of channels of the integrated perception signal input, the ECG signal is converted into multiple channels of approximate components and detail components through wavelet transform, wherein the approximate component corresponds to the integrated perception signal, that is, the low-frequency Information, detail components correspond to information entropy difference signals, that is, high-frequency signals. According to the correspondence between the above wavelet components and the integrated perception signals and the information entropy difference signals, these wavelet channels are combined in the data dimension. When the number of input channels of the integrated perception signal source is small, a shallow wavelet transform is used for preprocessing the original ECG signal. When the number of input channels of the integrated perception signal source increases, a deeper wavelet transform is used for preprocessing the original ECG signal. A data structure that meets the interface format requirements of the flow model reconstruction method is generated, which supports the input of variable-channel integrated perception signal sources and improves the accuracy and efficiency of ECG reconstruction.The processed data pairs are sent to the model construction unit 4084, which mainly initializes the basic operator layer of the ECG reconstruction model based on Flow and connects the operator layer, initializes the model training optimizer, defines the loss function, and encapsulates the data according to the format requirements of the reconstruction model reading. The constructed model, optimizer and data packet are passed to the training judgment unit 4085. The training judgment unit 4085 mainly determines whether it is necessary to train the reconstruction model. If it is necessary to train the model, the encapsulated, constructed model, optimizer and data packet are passed to the training unit 4086 for subsequent model training. The model training unit 4086 mainly uses the formatted ECG signal and the integrated perception signal data to train and adjust the weights of the reconstruction model, so that the reconstruction model can gradually output high-quality reconstructed ECG signals. After the model converges, the trained reconstruction model is passed to the ECG reconstruction reasoning unit 4087 for ECG reconstruction usage scenarios. The ECG reconstruction reasoning unit 4087 is mainly used to read the integrated perception signal and the insertion vector sampling module 4088 samples and generates the insertion vector from the preset information entropy difference probability distribution between the reconstructed source signal (integrated perception signal) and the ECG signal, and outputs a high-fidelity reconstructed ECG for health monitoring.
[0104] The following conclusion Figure 7 and Figure 8 The ECG reconstruction process is explained from the application perspective. In this embodiment, the ECG reconstruction method based on Flow is demonstrated by taking the joint acquisition of BCG and PPG, inputting into the ECG reconstruction model, and then outputting the reconstructed ECG as an example.
[0105] like Figure 7 As shown, the ECG acquisition terminal 401 is deployed with a signal acquisition terminal 400 for collecting ECG signals and responsible for signal transmission; the PPG smart watch 402 is deployed with a signal acquisition terminal 400 for collecting PPG signals and responsible for signal transmission; the BCG acquisition scale 403 is deployed with a signal acquisition terminal 400 for collecting BCG signals and responsible for signal transmission; the 404 laptop computer and the 405 smart phone are responsible for deploying a data management module 406, storing a data set 407 (that is, a database) and running an ECG reconstruction module 408. In general scenarios, especially in cases involving daily health monitoring, exercise status recording, physiological index monitoring during work, etc., the ECG acquisition terminal 401 does not have the characteristics of convenient, non-interference, and effective application, because in these scenarios, users need to carry out other activities while wearing health monitoring equipment, such as fitness, running, work, etc. If we want to obtain high-quality ECG signals in these scenarios, we need to first build a complete ECG reconstruction system, and then only wear the smart watch 402 and use the BCG acquisition scale 403 in the application to obtain high-quality ECG signals.
[0106] The data acquisition unit in the ECG acquisition terminal 401 includes ECG acquisition electrodes, operational amplifiers, and ADC analog-to-digital converters, which collect ECG electrocardiogram data of the current wearer. The ECG raw data is sent to the data processing unit in the ECG acquisition terminal 401 for further processing. The data processing unit in the ECG acquisition terminal 401 uses a FIR filter with a cutoff frequency of 0.3Hz to 40Hz according to the signal spectrum distribution of ECG, and filters out the baseline drift caused by the disturbance of the wire and the high-frequency noise caused by the surrounding environment. The filtered ECG signal is sent to the data transmission unit in the ECG acquisition terminal 401. The data transmission unit in the ECG acquisition terminal 401 selects Bluetooth as the wireless transmission method to transmit the digitized ECG signal to the 404 laptop computer for data aggregation and subsequent use.
[0107] The data acquisition unit in the PPG acquisition smartwatch uses a PPG sensor to sense the PPG data of the current wearer, and the raw PPG data is sent to the data processing unit in the PPG acquisition smartwatch for further processing. The data processing unit in the PPG acquisition smartwatch, according to the signal spectrum distribution of PPG, currently uses a Butterworth filter with a cutoff frequency of 1Hz to 8Hz to filter out the baseline drift caused by the wire disturbance and the high-frequency noise caused by the surrounding environment, and the filtered PPG signal is sent to the data transmission unit in the PPG acquisition smartwatch. The data transmission unit in the PPG acquisition smartwatch preferably uses Bluetooth as a wireless transmission method to transmit the digitized PPG signal to the 404 laptop computer for data aggregation and subsequent use.
[0108] The data acquisition unit in the BCG collection scale uses slight changes in the load sensor to sense the BCG data of the current subject. The BCG raw data is sent to the data preprocessing unit in the BCG collection scale. According to the signal spectrum distribution of BCG, the current filter uses an FIR filter with a cutoff frequency of 0.5Hz to 20Hz to filter out the noise in the BCG signal. The filtered BCG signal is sent to the data transmission unit in the BCG collection scale, and Bluetooth is preferably used as the wireless transmission method to transmit the digitized BCG signal to the 404 laptop computer for data aggregation and subsequent use.
[0109] In this embodiment, the laptop computers 404 and 405 are responsible for the deployment of the data management module 406, the storage of the data set 407, and the operation of the ECG reconstruction module 408. The data management module 406 uses a Bluetooth receiver to collect ECG, PPG, and BCG data from 401, 402, and 403, and synchronizes the data of the three channels, and then sends them to the data management unit 4061 for standardized data storage and management. 407 uses a MySQL database for data storage and management, and updates the 407 data set for ECG reconstruction model training. The ECG reconstruction module 408 includes a data reading unit 4081, a data preprocessing unit 4082, a signal type judgment unit 4083, a model building unit 4084, a training judgment unit 4085, a model training unit 4086, and an ECG reconstruction reasoning unit 4087. The data reading unit 4081 is a software running in a laptop computer for reading ECG, PPG and BCG signal data sets from a memory, which can convert the signal data from a binary format into a data structure that can be processed by the software, and then send it to the data preprocessing unit 4082 for subsequent data preprocessing, wherein it is mainly used to convert the original The ECG, PPG and BCG data are processed into the format required by the model input. First, the ECG, PPG and BCG waveforms are referred to the corresponding relationship between the cardiophysiological indicators, and the ECG R wave, the PPG trough point and the BCG J wave are aligned in the time domain; then the ECG, PPG and BCG signals are divided into heartbeat-level data according to the above alignment points and form data pairs, and normalized transformation is performed according to the heartbeat-level data segments; the signal type judgment unit 4083 determines according to the adaptive method of the number of reconstructed source channels, and the current embodiment uses 2 integrated sensing signals, and performs a secondary wavelet transform on the ECG channel, ECG where φ a,b and ψ a,bRespectively represent the approximate function and detail function of the Bb5 wavelet basis function with a scale of a and a translation of b, N represents the length of the ECG signal, in this embodiment, the translation of the wavelet transform is 0, and the scales of the transform are 1 and 2 respectively, firstly, the original ECG signal is used as input, and a transform with a transform scale of 1 is performed, and the approximate component and the first-level detail component are output, then the approximate component is used as input, and a transform with a transform scale of 2 is performed, and the second-level approximate component and the second-level detail component are output, and the second-level approximate component, the first-level detail component and the second-level detail component are interpolated with zero values, expanded to ECG equal-length data, and wavelet reconstruction is performed, and finally converted into ECG data consisting of three channels of the second-level approximate component, the first-level detail component and the second-level detail component, and a data structure that meets the requirements of the reconstruction model interface format is output and sent to the model construction unit 4084. The model construction unit 4084 mainly initializes the basic operator layer of the ECG reconstruction model based on Flow and connects the operator layer. The model consists of multiple Flow model basic building blocks TimeFlow and 1x1conv convolutional layers. Let x ECG 、x BCG 、x PPG denote the ECG signal pathway, the BCG signal pathway, and the PPG signal pathway, respectively. Let z denote the interpolation information pathway, wherein z~p(z) is the preset probability distribution of the interpolation information. In this embodiment, Gaussian distribution is selected as the preset probability distribution. x ECG ~p(x ECG |x BCG , x PPG )ECG signal is a conditional probability distribution that depends on BCG signal and PPG signal. Let Represent the reconstructed ECG signal, the synthesized BCG signal pathway, and the synthesized PPG signal pathway respectively. The ECG reconstruction model based on Flow converts x ECG Map to As well as the interpolation vector z, in the reverse reasoning process, the interpolation vector z randomly sampled from the preset Gaussian distribution and the original x BCG 、x PPG Refactoring The forward propagation process can be expressed as The reverse reasoning process can be expressed as In this embodiment, θ It consists of multiple stacked reversible basic blocks f i The flow of ECG data in these basic blocks can be described as: where h iis the temporary output of the TimeFlow basic block in the middle layer of the model, L is the number of basic blocks stacked in the flow model, the more basic blocks are stacked, the stronger the representation ability of the model is, and the more time the model training and reasoning process consumes. In this embodiment, it is set to 50, which can achieve high-quality ECG reconstruction and maintain a reasonable reasoning time. The reverse reasoning process of the reconstruction model based on Flow can be described as: where f i -1 It represents the reverse derivation of the TimeFlow basic block, which ensures the reversibility of the model. The mapping relationship between ECG, BCG and PPG is constructed within the reversibility framework to meet the correlation of the three signals in the perception principle. ECG signals perceive the electrical activity signals of the heart. Both PPG and BCG perceive the changes in body organs caused by the beating of the heart. The source of the changes in body organs is also the electrical activity of the heart. The data flow process of the TimeFlow basic block can be expressed as: Among them, F(.), H(.), and G(.) are three sub-neural networks with the same structure used to construct TimeFlow, which are mainly composed of recurrent neural operators and fully connected operators. In this embodiment, the recurrent neural operator is a long short-term memory unit (LSTM). The starting operator of the sub-neural network is a recurrent neural operator with a hidden unit dimension of 200, followed by three cascaded fully connected operators. The number of units of the first and second level fully connected operators is 200, and the number of units of the last level fully connected operator is 125, which is consistent with the dimension of the heartbeat-level data end of ECG, BCG, and PPG. After the model is built, the system configures the optimizer used for model training. In this embodiment, the Adam optimizer is configured, and the learning rate parameter is configured to 0.0005, and the weight attenuation parameter is 0.0001. The loss function of the reconstruction model based on Flow in this embodiment consists of three parts: supervision loss, that is, in the process of forward reasoning of the model, the output synthesis of the model PPG and BCG is: It should be close to the original PPG and BCG in waveform; reconstruction loss, in order to generate high-quality reconstructed ECG, the reconstructed ECG output by the Flow-based reconstruction model during the reverse reasoning process is It should be consistent with the original ECG approximation; Stable loss (also called information entropy difference loss), in order to enable the Flow-based reconstruction model to stably update the model gradient during the training process, the cross entropy of the information entropy difference vector is used to construct a stable loss (also called information entropy difference loss). The total loss function used by the system is expressed as: where ω 1 ,ω 2 ,ω 3is the weight coefficient constant, q(ECG) is the distribution of input ECG, p(z) is the probability distribution of information entropy difference vector z, The value of z is The probability of , where the third stability loss (also called information entropy difference loss) represents The similarity between the predicted distribution of z and the true distribution of z, E q(ECG) It represents the expected similarity between the probability of z output by the flow model inference and the actual probability of z when all ECGs observed under the condition of unknown ECG probability distribution q(ECG) are used as input. The total loss function used by the system is composed of the root mean square error between the reconstructed ECG and the original ECG, the root mean square error between the synthetic PPG and the original PPG, the synthetic BCG and the original BCG, and the cross entropy of the information entropy difference vector. The constructed ECG reconstruction model, optimizer, loss function and data packet are transmitted to the training judgment unit 4085. If the training judgment unit 4085 determines that training is needed, the model training unit 4086 performs subsequent model training.
[0110] The model training unit 4086 first divides the ECG, BCG and PPG data sets after preprocessing into training sets and test sets. The present embodiment divides them into training sets and test sets in a ratio of 8:2. Using the training data set as input, the Flow-based reconstruction model first performs forward process propagation, outputs synthesized BCG and synthesized PPG and predicted interpolation vectors, and the system records the synthesized BCG and synthesized PPG and predicted interpolation vectors. Subsequently, the original BCG and PPG and interpolation vectors are used as input, and the Flow-based reconstruction model performs reverse process propagation, and outputs reconstructed ECG. The system uses the synthesized BCG, synthesized PPG, interpolation vectors and reconstructed ECG to calculate the loss function, and then the system calculates the weight update gradient of the reconstruction model based on the loss amount, and the optimizer uses the gradient information to update and adjust the weights. The above process is the training process of the model. After the model training is completed, the training judgment unit 4085 uses the divided test set to evaluate the model. If the model converges, the model training is stopped, and the model and the updated model weights are output to the ECG reconstruction reasoning unit 4087 for ECG reconstruction usage scenarios; if the model does not converge, the above model training process continues. In this embodiment, the ECG reconstruction reasoning unit 4087 mainly reads the BCG and PPG signals and the insertion vector sampling module 4088 obtains the interpolation vector based on the preset interpolation vector distribution function sampling as the input of the ECG reconstruction reasoning unit 4087, performs model reasoning, and outputs a high-fidelity reconstructed ECG for health monitoring.
[0111] like Figure 6 As shown, a preferred execution process of an ECG reconstruction method is:
[0112] Step 601: The ECG acquisition terminal, PPG acquisition smart watch, and BCG acquisition scale use their own integrated sensors to respectively collect ECG, PPG, and BCG data, filter the data, and then transmit the data to the data management module on the computer.
[0113] Step 602: The data management module on the computer receives data from the ECG acquisition terminal, the PPG acquisition smart watch, and the BCG acquisition scale and stores the collected data in the memory on the computer.
[0114] Step 603: Determine whether the electrocardiogram reconstruction model needs to be trained, if so, execute step 604, if not, execute step 611.
[0115] Step 604: Determine whether the amount of data in the data set stored on the computer is sufficient for training the reconstruction model. If the amount of data is not sufficient, return to step 601 to continue collecting ECG, PPG, and BCG data; if the amount of data is sufficient, proceed to step 604 for subsequent data processing.
[0116] Step 605: The ECG reconstruction module on the computer reads the stored data set, and pre-processes the ECG, PPG, and BCG data respectively, and finally outputs data that meets the input format requirements of the electrocardiogram reconstruction model.
[0117] Step 606: The system divides the pre-processed data samples into a training set and a test set according to a ratio of 8:2.
[0118] Step 607: construct an ECG reconstruction model based on Flow, randomly initialize its weights, construct an Adam optimizer, and set the learning rate and weight decay parameters.
[0119] Step 608: Using the training set as input, first take the original ECG as input, the ECG reconstruction model performs forward process reasoning, and outputs synthetic PPG and BCG; then take the original BCG and the original PPG as input, the ECG reconstruction model performs reverse process reasoning, and outputs reconstructed ECG; use the above output and the original data to input into the loss function calculation formula, calculate the loss value, use the loss value to calculate the update gradient of the model, and update the weights of the reconstruction model.
[0120] Step 609: using the test set to test whether the ECG reconstruction model has converged, if it has converged, proceed to step 610; if it has not converged, return to step 608 to continue training the ECG reconstruction model;
[0121] Step 610: The Flow-based reconstruction model structure and the updated ECG reconstruction model weight data are stored on the computer.
[0122] Step 611: Enter the model deployment process. First, the ECG reconstruction model structure and ECG reconstruction model weight data stored on the computer are sent to the mobile phone. After the mobile phone collects the PPG and BCG data, it uses the model to reconstruct the ECG inference.
[0123] Step 612: The PPG collection smart watch collects PPG signals and sends them to the mobile phone via Bluetooth. The BCG collection scale collects BCG signals and sends them to the mobile phone via Bluetooth. The data management module on the mobile phone aggregates the PPG data and the BCG data.
[0124] Step 613: The ECG reconstruction module on the mobile phone reads the PPG and BCG data and performs data preprocessing to process the PPG and BCG data into a data structure that meets the input requirements of the electrocardiogram reconstruction model.
[0125] Step 614: The mobile terminal reads the Flow-based reconstruction model structure and model weight data, and uses the PPG and BCG data as input to start reasoning of the reconstruction model, and finally outputs the reconstructed ECG (see Figure 8 ).
[0126] In summary, the present invention proposes a complete ECG reconstruction system, which includes the data set acquisition process required for the construction of the ECG reconstruction method, the training of the ECG reconstruction model, and the deployment of the model, providing a complete and feasible ECG reconstruction process. The ECG reconstruction system of the present invention provides a complete set of data set acquisition processes, including the selection of signal acquisition equipment, the setting of acquisition parameters, the preprocessing of acquired data, and other steps to ensure the quality and reliability of the data set. The ECG reconstruction system of the present invention uses a technical framework based on the Flow flow model to construct an ECG reconstruction model. The training process includes the division of the data set, the implementation of the model structure, the setting of the loss function for the Flow flow model, the design of the model's hyperparameters, and other steps, which can improve the accuracy and generalization performance of the model. The ECG reconstruction system of the present invention provides a complete set of model deployment processes, including the distribution of the model, the data flow design of the deployment environment, and other steps to ensure that the model can run stably in the actual environment.
[0127] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.
[0128] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.
[0129] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.
[0130] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A training method for an electrocardiogram reconstruction model, characterized in that: The method comprises: S1. Obtain a training data set consisting of a plurality of training data pairs, wherein each training data pair includes ECG signal conversion data and immersive perception signal data of a user at the same time, wherein the ECG signal conversion data is obtained by performing channel number conversion on the ECG signal data collected by the user according to the channel number of the immersive perception signal data, so that the channel number of the ECG signal conversion data is the channel number of the immersive perception signal data plus 1; S2, obtaining an electrocardiogram reconstruction model, which is a flow model; S3. Using the training data set to train the electrocardiogram reconstruction model, wherein the training includes a process of forward reasoning and reverse reasoning. During forward reasoning, the electrocardiogram reconstruction model is used to generate synthetic integrated perception signal data and an information entropy difference vector based on the ECG signal conversion data, and the information entropy difference vector indicates the information entropy difference between the ECG signal conversion data and the integrated perception signal data. During reverse reasoning, the electrocardiogram reconstruction model is used to generate synthetic ECG signal conversion data based on the integrated perception signal data and an interpolation vector randomly sampled from a preset probability distribution of the information entropy difference information, and the parameters of the electrocardiogram reconstruction model are updated using the information entropy difference loss corresponding to the information entropy difference vector.
2. The method according to claim 1, characterized in that The information entropy difference loss is determined based on the similarity between the probability distribution of the information entropy difference vector of each training data and a preset Gaussian probability distribution; or, the information entropy difference loss is equal to the mean of the information entropy difference vectors obtained for each training data pair.
3. The method according to claim 2, characterized in that During training, the parameters of the ECG reconstruction model are updated based on the total loss determined by the information entropy difference loss, the reconstruction loss between the synthesized ECG signal conversion data and the ECG signal conversion data, and the supervision loss between the synthesized immersion perception signal data and the immersion perception signal data.
4. The method according to claim 3, characterized in that: The immersive sensing signal data includes ballistocardiogram signal data and photoplethysmogram signal data, and the total loss is determined in the following manner: Among them, L total represents the total loss, ω1 represents the first hyperparameter, ω2 represents the second hyperparameter, and ω3 represents the third hyperparameter. represents the reconstruction loss, represents the synthetic ECG signal conversion data, x ECG Represents ECG signal conversion data, represents the supervision loss, represents the first supervision loss corresponding to the ballistocardiogram signal data, represents the synthesized ballistocardiogram signal data, x BCG represents the ballistocardiogram signal data, represents the second supervision loss corresponding to the photoplethysmography signal data, represents the synthesized photoplethysmography signal data, x PPG represents the photoplethysmographic signal data, and T represents the information entropy difference loss.
5. The method according to claim 1, characterized in that The channel number conversion is performed as follows: Perform multiple wavelet transforms on ECG signal data, and obtain an approximate component and a detail component each time; Zero-value interpolation is performed on the approximate component and detail component obtained by the last wavelet transform and the detail component of the wavelet transform before the last wavelet transform to obtain ECG signal conversion data.
6. The method according to any one of claims 1 to 5, characterized in that: The electrocardiogram reconstruction model is provided with at least one recurrent neural network for extracting time domain features and at least one fully connected neural network for extracting space domain features.
7. An electrocardiogram reconstruction method based on an electrocardiogram reconstruction model, characterized in that: The method comprises the following steps: C1. Acquire historical data of the user, wherein the historical data includes ECG signal data and immersive sensing signal data of the user at multiple times; C2, preprocessing the physiological signal obtained in step C1 according to a preset processing method to obtain training starting data consisting of a plurality of data pairs, wherein each data pair includes ECG signal data and immersive sensing signal data of the user in at least one heartbeat cycle; C3, adding 1 to the number of channels of the integrated perception signal data in step C2 as the number of conversion channels; C4, performing channel number conversion on the ECG signal data in the data pair obtained in step C2 to obtain ECG signal conversion data, wherein the ECG signal conversion data and the immersive sensing signal data in one data pair form a training data pair, and a plurality of training data pairs form a training data set; C5. Training the electrocardiogram reconstruction model multiple times using the training method described in any one of claims 1 to 10 to obtain a trained electrocardiogram reconstruction model; C6. Obtain the user's current immersive perception signal, pre-process the user's current immersive perception signal according to a preset processing method to obtain the user's current immersive perception signal data, and generate an insertion vector by sampling from a preset information entropy difference information probability distribution between the immersive perception signal and the ECG signal; C7, inputting the user's current immersive perception signal data and the insertion vector generated by sampling into the trained electrocardiogram reconstruction model to obtain the user's current reconstructed ECG signal conversion data; C8. Perform an inverse channel transformation on the user's current reconstructed ECG signal conversion data to obtain reconstructed ECG signal data.
8. The method according to claim 7, characterized in that The preset processing method includes one or more of the following methods: filtering, alignment, segmentation, and normalization.
9. An electrocardiogram reconstruction system, characterized in that: The system comprises: An information collection terminal, which is configured to collect physiological signals of a user, and perform denoising on the collected physiological signals and output them; wherein the physiological signals include ECG signals and immersive sensing signals; A data management module, which is configured to receive the data output by the information collection terminal and process it according to a preset data structure; A database, configured to store the data processed by the data management module; A reconstruction module is configured to execute the method according to any one of claims 7-8 to obtain reconstructed ECG signal data.
10. The system according to claim 9, characterized in that The reconstruction module includes: A data reading unit, which is configured to: acquire and output a training physiological signal of a detection target, wherein the training physiological signal is an ECG signal and an immersive sensing signal of the detection target in a plurality of consecutive heartbeat cycles; A data preprocessing unit, which is configured to: receive the physiological signal output by the data reading unit, perform preprocessing according to a preset processing method, obtain and output training starting data composed of data pairs of the detection target in each heartbeat cycle, wherein each data pair is ECG signal data and immersive sensing signal data of the detection target in the same heartbeat cycle; A signal type determination unit, configured to: add 1 to the channel number of the immersive sensing signal data as the conversion channel number, and convert the ECG signal data into ECG signal conversion data, wherein the channel number of the ECG signal conversion data is equal to the conversion channel number; A model building unit, which is configured to: store a pre-initialized electrocardiogram reconstruction model; A training judgment unit, configured to: determine whether the electrocardiogram reconstruction model stored in the model building unit needs to be trained, and output a training judgment result; A model training unit, configured to: when the training judgment result output by the training judgment unit is that training is required, execute the training method according to any one of claims 1 to 6 to update the parameters of the model in the model building unit; An insertion vector sampling module is configured to: generate an insertion vector by sampling from a preset information entropy difference probability distribution between the immersive perception signal and the ECG signal; The reconstruction reasoning unit is configured as follows: when the training judgment unit outputs a result that no training is required, the electrocardiogram reconstruction model stored in the model building unit is used as the trained electrocardiogram reconstruction model, and the integrated perception signal in the physiological signal of the detection target is obtained and processed according to a preset processing method to obtain the integrated perception signal data and the insertion vector obtained by the insertion vector sampling module, and the trained electrocardiogram reconstruction model is used to reconstruct the electrocardiogram.
11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of any method described in claims 1 to 8.
12. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the steps of the method as claimed in any one of claims 1 to 8.
Citation Information
Patent Citations
Method and system for extracting electrocardiosignal characteristic
CN101828917A
Automated cardiac volume segmentation
CN108603922A
Electrocardiogram anomaly recognition method and device, computer equipment and storage medium
CN111759304A
Oven
KR1020240026782A
Method for generating a model for generating a synthetic ECG and a method and system for analysis of heart activity
US20200196897A1
Cited By
Electrocardio diagnosis model system based on AI
CN120241093A
