A method for electrocardiosignal compression sensing based on transformer

Abstract

The application discloses a kind of electrocardiosignal compression sensing methods based on Transformer, comprising the following steps: S1) data set making;S2) construct the compression sensing network based on Transformer;S3) construct final loss function;S4) train the compression sensing network based on Transformer;S5) compression sensing reconstruction effect evaluation, test data set is cut into training block same size data block, then it is sent into the neural network after training in step S4) and is carried out prediction, again quantitative index such as percentage root mean square error PRD, signal-to-noise ratio SNR is verified on test set.The application can quickly and accurately reconstruct the low-dimensional electrocardiosignal of compressed sampling by the ability of global characteristics captured by Transformer network and the advantage of processing sequence-to-sequence task, which not only reduces system power consumption, but also obtains high-quality reconstructed electrocardiosignal.

Description

Technical Field

[0001] The present invention belongs to the technical field of electrocardiogram monitoring, and in particular relates to a Transformer-based electrocardiogram signal compression sensing method. Background Art

[0002] Electrocardiogram (ECG) signals, a form of human medical information, are currently a key technology used clinically to detect and diagnose cardiovascular disease. In recent years, wireless human sensor networks have been widely used to obtain real-time human medical information. Long-term, real-time monitoring of ECG signals can help detect and prevent cardiovascular disease. However, long-term, real-time monitoring requires exponential growth in the amount of ECG signal data sampled and transmitted. Traditional sampling and transmission technologies, limited by the power consumption and lifespan of mobile devices, are unable to support long-term ECG monitoring. Therefore, effectively reducing ECG signal redundancy during the sampling and transmission process, and thus reducing the amount of data required to be transmitted and stored, is crucial to the implementation of these technologies.

[0003] Compressed sensing theory breaks the limitations of traditional sampling theory by leveraging signal sparsity to extract effective information from fewer samples, reducing the frequency and complexity of the sampling process and further lowering overall system power consumption. This provides a solution for long-term, real-time monitoring and low-power transmission of ECG signals. Compressed sensing constructs a suitable observation matrix to acquire low-dimensional measurement signals. This is then reconstructed stably and accurately using a reconstruction algorithm at the receiving end. The compressed sensing reconstruction algorithm determines whether a high-quality, high-precision reconstructed signal can be obtained.

[0004] Traditional compressed sensing reconstruction algorithms mainly utilize the sparsity and prior knowledge of the signal in the transform domain, and iteratively reconstruct the original signal by solving optimization problems. They have high computational complexity and time complexity and cannot complete the reconstruction task immediately. The compressed sensing method based on convolutional neural networks has shown good results, but the application limitations caused by the local connection and weight sharing characteristics of convolutional neural networks are also becoming increasingly prominent. Each neuron in the convolutional neural network only focuses on the local position information in the input sequence. This local focus characteristic may lead to insufficient capture of global information, thereby affecting the effective representation and reconstruction of long sequence signals. In order to solve the above problems and better complete the reconstruction task of compressed sensing of ECG signals, the present invention proposes a compressed sensing method for ECG signals based on Transformer. Summary of the Invention

[0005] Purpose of the invention: In order to overcome the above shortcomings, the purpose of the present invention is to provide a Transformer-based compressed sensing method for ECG signals. The Transformer network has the ability to capture global characteristics and the advantage of processing sequence-to-sequence tasks, and can quickly and accurately reconstruct the compressively sampled low-dimensional ECG signals, which not only reduces system power consumption, but also obtains high-quality reconstructed ECG signals, and can meet the excellent quality standards of the reconstructed signals at the medical level.

[0006] Technical solution: To achieve the above objectives, the present invention provides a Transformer-based ECG signal compression sensing method, comprising the following steps:

[0007] S1): Dataset preparation: collect and integrate the ECG data to be trained, form the ECG data into an experimental dataset, filter, de-noise and normalize the data in the experimental dataset, and then randomly divide it into a training dataset and a test dataset in a ratio of 8:2;

[0008] S2): Constructing a Transformer-based compressed sensing network, wherein the Transformer-based compressed sensing network includes a learnable sampling module, a preliminary reconstruction module, and a deep reconstruction module;

[0009] First, the sampling module uses a data-driven trainable perception matrix to compress and sample the initial ECG signal to obtain a low-dimensional compressed sampling signal.

[0010] Simulate the process of data transmission to the server terminal; then restore and reconstruct the signal through the two stages of preliminary reconstruction module and deep reconstruction module;

[0011] S3): Construct the final loss function, which consists of two parts: initial reconstruction loss and deep reconstruction loss;

[0012] S4): Train a Transformer-based compressed sensing network, i.e., divide the training dataset from step S1) into non-overlapping data segments and feed them into the neural network for training. The trained neural network is used for compressed sensing sampling and reconstruction of ECG signals. The training block size is 256×256 and the batch size is 32. The training process uses a polynomial learning strategy to adjust the learning rate. The number of training rounds is 200. Early stopping is used to avoid overfitting when the loss value calculated by the final loss function stops decreasing or oscillates.

[0013] S5): Evaluation of the compressed sensing reconstruction effect: cut the test dataset into data blocks of the same size as the training blocks, and then feed them into the neural network trained in step S4) for prediction. Quantitative indicators such as percentage mean square error (PRD) and signal-to-noise ratio (SNR) are then used to verify the results on the test set, and the diagnostic performance of different methods is compared.

[0014] In the Transformer-based ECG signal compressed sensing method of the present invention, the specific sampling method of the learnable sampling module in S2) is as follows:

[0015] First, the ECG signal is divided into C×L non-overlapping data segments, where C represents the number of ECG signal channels and L represents the length of the data segment.

[0016] Let Φ∈R M×N is the trainable perception matrix, M = σ × L, N = L;

[0017] The sampling rate σ is as follows:

[0018] σ=M / N(0≤σ≤1); the compressed sampling process can be expressed as:

[0019] y=S(x,Φ))=Φ·x

[0020] Where y is the compressed sampled ECG signal, x is the original ECG signal, and S(x,Φ) represents the compressed sampling operation;

[0021] The sampling module jointly trains the elements of the perception matrix Φ (initialized with a normal distribution) through backpropagation from training, and the elements of the perception matrix Φ ultimately follow a quasi-Gaussian distribution. After joint training with the subsequent reconstruction module, the sampling module adaptively participates in the architecture to improve signal reconstruction quality. In practical applications, the trained Φ is used to obtain measurement values ​​from the raw ECG signal.

[0022] In the Transformer-based ECG signal compression sensing method of the present invention, the specific process of restoring and reconstructing the signal in two stages in S2) is as follows:

[0023] The first stage preliminary reconstruction module uses the data-driven initial reconstruction matrix To initially restore the signal by increasing the dimension, the initial reconstruction module uses the characteristics of the ECG signal learned during the training process to perform preliminary reconstruction of the signal, using the trainable initial reconstruction matrix Initial restoration signal; the initial reconstruction matrix Initialized to the transpose of the sampling matrix Φ, and automatically trained through back propagation, the elements in the matrix are trained jointly by back propagation and reconstruction modules to fully learn the prior information of the ECG signal. The signal is preliminarily reconstructed using the features of the ECG signal learned during training to obtain the initial reconstructed signal.

[0024] The second-stage deep reconstruction module uses a Transformer-based hybrid reconstruction network to restore and reconstruct the signal, and uses a preprocessing convolutional network to learn the local details between ECG signals. The hybrid reconstruction network embeds the preprocessing results and the initial reconstructed signal into a data representation and adds position information as the input sequence of the Transformer. The original ECG signal is used as the target sequence, and the encoder and decoder of the Transformer are used to extract the data features of the ECG signal and generate a reconstructed ECG signal. Through a customized Transformer backbone network and CNN reconstruction module, the Transformer-based ECG signal compression sensing network can effectively capture the global positional dependencies and local details between signals. At the same time, using an optimizer, it effectively establishes a mapping relationship between the original signal and the predicted signal, thereby obtaining a high-performance reconstructed ECG signal.

[0025] The Transformer-based ECG signal compression sensing method of the present invention, the initial reconstruction module in S2) uses the features of the ECG signal learned during the training process to perform preliminary reconstruction of the signal, using a trainable initial reconstruction matrix To restore the signal initially, Initialized as the transposed matrix of the sampling matrix Φ and automatically trained through back propagation, for a sampling rate σ = M / N, the number of rows of the measurement signal y obtained by compression sampling of the sampling module is σ × L;

[0026]

[0027] Among them, x init is the initial reconstructed ECG signal, represents the initial reconstruction process, and y = S(·,Φ) is the compressed sampling signal.

[0028] The Transformer-based ECG signal compression sensing method of the present invention comprises a preprocessing convolutional network comprising an input layer, a two-dimensional convolutional layer, a batch normalization layer, an ELU activation layer, and an output layer connected in sequence;

[0029] The process of learning local details of the signal through the preprocessing convolutional network can be expressed as:

[0030] x pre =x ini -C pre (x ini )

[0031] Among them, C pre (·) represents the preprocessing convolution layer, x pre Represents the signal after preprocessing convolution.

[0032] The Transformer-based ECG signal compression sensing method of the present invention comprises a Transformer backbone network composed of a pair of encoders and decoders; the initial reconstructed signal and the preprocessed ECG signal are respectively position-coded after being embedded and then input into the encoder-decoder;

[0033] The encoder is composed of multiple identical layers stacked together, each of which consists of two sublayers: a permuted multi-head self-attention layer and a feedforward network layer, where each sublayer uses residual connections and layer normalization;

[0034] The encoder process in the Transformer backbone network of the depth reconstruction module can be expressed as:

[0035] x en =T en (EM(x ini )+PE(x ini ))

[0036] Among them, T en (·) represents the encoder of Transformer, EM(·) represents the embedding layer, and init For vectorized representation, PE(·) represents the matrix function that measures the correlation of input positions;

[0037] The decoder is composed of multiple identical layers stacked together, each of which consists of three sublayers: a permuted multi-head self-attention layer, a permuted multi-head cross-attention layer, and a feedforward network layer. Residual connections and layer normalization are used in the layers.

[0038] The process of the decoder in the Transformer backbone network of the depth reconstruction module can be expressed as:

[0039] x de =T de (x en ,EM(x pre ))

[0040] Among them, T de (·) represents the decoder process of Transformer, which is similar to the encoder T en (·), but the decoder input contains the encoder output x en And the output of the embedding layer EM(x pre ), and finally, the output ECG signal data are combined to form the reconstructed ECG signal data.

[0041] The Transformer-based ECG signal compression sensing method of the present invention is characterized in that each head in the rearranged multi-head self-attention layer is a scaled dot product attention mechanism combined with a depthwise separable convolution, so that each head can focus on different parts of the input sequence and calculate different weighted sums. The scaled dot product attention mechanism obtains Q, K, and V by linearly projecting the input, where K and V are rearranged by depthwise separable convolution to reduce the spatial dimension and obtain K. p , V p ; Q, K p The weights are generated through dot product operations and finally act on V p It visits any position of the signal at a constant cost and pays more attention to positions with higher attention weights, so that each head can effectively reduce complexity while focusing on different parts of the input sequence;

[0042] The scaled dot product attention mechanism is implemented by Perform linear projection to obtain Q, K, V, with dimensions d respectively. q d k and d v , the projection process is as follows:

[0043] Q,K,V=F qkv (x in );

[0044] Input signal x in After h independent linear projections, we get h groups of dimensions d q , d k and d v The Q, K, and V of ​​correspond to the input of h attention heads respectively:

[0045]

[0046] The process of rearranged multi-head self-attention operation is as follows:

[0047]

[0048] Among them, Dconv(·) represents the depth-separable convolution operation, which reduces the channel dimension of K and V to d model / 4, to achieve feature rearrangement;

[0049] The operation can be specifically expressed as:

[0050]

[0051] in, For the permuted multi-head self-attention mechanism, x in=EM(x ini )+PE(x ini );

[0052] The matrix of linear projection is

[0053] The structure of the reordered multi-head cross attention layer is exactly the same as that of the reordered multi-head self-attention layer. Therefore, the reordered multi-head cross attention mechanism has the same operation process as the reordered multi-head self-attention. The difference is that its scaled dot product attention mechanism is implemented by the encoder output x en Perform linear projection to obtain Q, K, and calculate EM(x pre ) Perform linear projection to obtain V;

[0054] The details are as follows:

[0055] Rearrange the multi-head cross attention mechanism Expressed as:

[0056]

[0057] The Transformer-based ECG signal compressed sensing method of the present invention, the scaled dot product attention mechanism calculates the dot product of Q and all K in the projection process and divides each value by the scale factor To obtain weight;

[0058] The Softmax function is then applied, where the scaling factor Helps produce smoother Softmax output;

[0059] Finally, the attention weights are applied to V;

[0060] To do this, the query, keys, and values ​​are packed into a matrix

[0061] Since the self-attention mechanism can access features at any position at a constant cost, it can capture the dependencies between different heartbeat cycles in the same signal segment, thereby generating more representative features for more accurate signal reconstruction.

[0062] In the Transformer-based ECG signal compression sensing method of the present invention, the final loss function constructed in S3) is specifically as follows:

[0063] The initial reconstruction loss L of the initial reconstruction module ini ,

[0064] Use x ini The cosine similarity between x and x is measured as a loss function; the depth reconstruction loss L for the depth reconstruction modulecon ,This study uses Euclidean distance to calculate,

[0065] Therefore, the total loss L is: L = L ini +αL con

[0066] Among them, α is a weighting coefficient used to smooth the contribution differences between loss functions.

[0067] In the Transformer-based ECG signal compressed sensing method described in the present invention, the feedforward network layer consists of two fully connected layers. The output of the multi-head attention sublayer and the initial reconstructed signal are used as the input of the feedforward network after residual connection and layer normalization. The operation of the feedforward network can be expressed as:

[0068] x f =LN(x mhsa +x in )

[0069] x ffn =LN(ReLU(x f )+x f )

[0070] Among them, x f is the input of the feedforward network, x ffn is the output of the feedforward network, ReLU() is a rectified linear function, and LN stands for layer normalization.

[0071] In the Transformer-based ECG signal compressed sensing method of the present invention, step S5) uses the percentage mean square error (PRD) and signal-to-noise ratio (SNR) quantitative indicators to verify and compare the performance of neural networks of different methods on a test set. The specific contents are as follows:

[0072] PRD is the ratio of the l2 norm of the difference between the original signal and the reconstructed signal to the l2 norm of the original signal, which describes the reconstruction error. The smaller the PRD, the better the reconstruction quality.

[0073] The formula for PRD is:

[0074]

[0075] The signal-to-noise ratio is related to the ratio of the original signal to the noise, and is also used to describe the quality of reconstruction. The larger the SNR, the better the reconstruction effect.

[0076] The SNR formula is as follows:

[0077]

[0078] Among them, x and are the original ECG signal and the reconstructed signal respectively.

[0079] It can be seen from the above technical solution that the present invention has the following beneficial effects:

[0080] 1. The Transformer-based compressed sensing method for ECG signals described in the present invention combines compressed sensing and Transformer to propose a new compressed sensing sampling and reconstruction algorithm for ECG signals. It utilizes the Transformer network's ability to capture global characteristics and its advantage in processing sequence-to-sequence tasks, and can quickly and accurately reconstruct the compressively sampled low-dimensional ECG signals, thereby reducing system power consumption and obtaining high-quality reconstructed ECG signals. Furthermore, it can meet the excellent quality standards of the reconstructed signals at the medical level.

[0081] 2. The data-driven sampling mode of the present invention is different from the traditional compressed sensing algorithm which usually uses a fixed sensor matrix for random sampling. The learnable sampling module designed in the present invention can learn the prior knowledge of ECG signals during training, making full use of the ECG signal characteristics.

[0082] 3. The end-to-end hybrid architecture ECG signal compressed sensing algorithm proposed in this invention integrates the Transformer backbone architecture and convolutional neural network assistance, fully utilizing the advantages of Transformer and convolutional neural network to obtain high-quality ECG signal reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is a flow chart of the ECG-TransCS algorithm of the present invention;

[0084] Figure 2 Schematic diagram of the Transformer backbone structure in the depth reconstruction module of the present invention;

[0085] Figure 3 Schematic diagram of the preprocessing convolutional network in the deep reconstruction module of the present invention;

[0086] Figure 4 Schematic diagram of the multi-head attention mechanism in the present invention;

[0087] Figure 5 For the ECG of the present invention, the verification experiment of the present invention selected a signal segment of approximately 5 seconds from the 102nd record in the test dataset, compressed the sampling at a sampling rate of 0.1, and reconstructed the segment; (a)-(f) correspond to the reconstruction results of OMP, SP, BSBL-BO, CAE, CsNet, and the ECG-TransCS proposed in the present invention, respectively;

[0088] Figure 6 This is an experimental effect diagram of the present invention;

[0089] Figure 7 For the signal quality perceived in the present invention, different PRD values ​​are classified into different quality level comparison tables. DETAILED DESCRIPTION

[0090] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0091] Example

[0092] As shown in the figure, a Transformer-based ECG signal compressed sensing method includes the following steps:

[0093] S1): Dataset preparation: collect and integrate the ECG data to be trained, form the ECG data into an experimental dataset, filter, de-noise and normalize the data in the experimental dataset, and then randomly divide it into a training dataset and a test dataset in a ratio of 8:2;

[0094] S2): Constructing a Transformer-based compressed sensing network, wherein the Transformer-based compressed sensing network includes a learnable sampling module, a preliminary reconstruction module, and a deep reconstruction module;

[0095] First, the sampling module uses a data-driven trainable perception matrix to compress and sample the initial ECG signal to obtain a low-dimensional compressed sampling signal.

[0096] The process of simulating data transmission to the server terminal; then, the signal is restored and reconstructed through two stages: the preliminary reconstruction module and the deep reconstruction module. Specifically, the deep reconstruction module, consisting of a Transformer backbone network and an auxiliary CNN, uses a Transformer-based ECG signal compression sensing network to effectively capture global positional dependencies and local details between signals, thereby obtaining high-performance reconstructed ECG signals.

[0097] S3): Construct the final loss function, which consists of two parts: initial reconstruction loss and deep reconstruction loss;

[0098] S4): Train a Transformer-based compressed sensing network, i.e., divide the training dataset from step S1) into non-overlapping data segments and feed them into the neural network for training. The trained neural network is used for compressed sensing sampling and reconstruction of ECG signals. The training block size is 256×256 and the batch size is 32. The training process uses a polynomial learning strategy to adjust the learning rate. The number of training rounds is 200. Early stopping is used to avoid overfitting when the loss value calculated by the final loss function stops decreasing or oscillates.

[0099] S5): Evaluation of the compressed sensing reconstruction effect: cut the test dataset into data blocks of the same size as the training blocks, and then feed them into the neural network trained in step S4) for prediction. Quantitative indicators such as percentage mean square error (PRD) and signal-to-noise ratio (SNR) are then used to verify the results on the test set, and the diagnostic performance of different methods is compared.

[0100] In the Transformer-based ECG signal compressed sensing method described in this embodiment, the specific sampling method of the learnable sampling module in S2) is as follows:

[0101] First, the ECG signal is divided into C×L non-overlapping data segments, where C represents the number of ECG signal channels and L represents the length of the data segment.

[0102] Let Φ∈R M×N is the trainable perception matrix, M = σ × L, N = L;

[0103] The sampling rate σ is as follows:

[0104] σ=M / N(0≤σ≤1); the compressed sampling process can be expressed as:

[0105] y=S(x,Φ))=Φ·x

[0106] Where y is the compressed sampled ECG signal, x is the original ECG signal, and S(x,Φ) represents the compressed sampling process;

[0107] The sampling module jointly trains the elements of the perception matrix Φ (initialized with a normal distribution) through backpropagation from training, and the elements of the perception matrix Φ ultimately follow a quasi-Gaussian distribution. After joint training with the subsequent reconstruction module, the sampling module adaptively participates in the architecture to improve signal reconstruction quality. In practical applications, the trained Φ is used to obtain measurement values ​​from the raw ECG signal.

[0108] In the Transformer-based ECG signal compression sensing method described in this embodiment, the specific process of restoring and reconstructing the signal in two stages in S2) is as follows:

[0109] The first stage uses the data-driven initial reconstruction matrix To initially restore the signal by increasing the dimension, the initial reconstruction module uses the characteristics of the ECG signal learned during the training process to perform preliminary reconstruction of the signal, using the trainable initial reconstruction matrix Initial restoration signal; the initial reconstruction matrix Initialized to the transpose of the sampling matrix Φ, and automatically trained through back propagation, the elements in the matrix are trained jointly by back propagation and reconstruction modules to fully learn the prior information of the ECG signal. The signal is preliminarily reconstructed using the features of the ECG signal learned during training to obtain the initial reconstructed signal.

[0110] In the second stage, the mapping relationship between ECG signals and compressed sampling signals is learned through a Transformer-based hybrid reconstruction network using a Transformer-based deep reconstruction module. The dependencies between heartbeats in long sequence input signals are captured through the Transformer's permuted multi-head self-attention mechanism, and the local details between ECG signals are learned using a preprocessing convolutional network. The preprocessing results and the initial reconstructed signals are embedded into a data representation and position information is added as the input sequence of the Transformer. The original ECG signal is used as the target sequence to effectively establish a mapping relationship between the original signal and the low-dimensional sampling signal. The Transformer backbone network is used to extract the data features of the ECG signal and generate a reconstructed ECG signal, thereby accurately reconstructing the original signal through the low-dimensional sampling signal. At the same time, an optimizer is used with the mean square error as the loss function to train the model, evaluate the trained model, and calculate the recovery accuracy of the ECG signal.

[0111] It should be noted that the ECG signal is learned through the Transformer network.

[0112] In the Transformer-based ECG signal compressed sensing method described in this embodiment, the initial reconstruction module in S2) uses the features of the ECG signal learned during the training process to perform preliminary reconstruction of the signal, using a trainable initial reconstruction matrix To restore the signal initially, Initialized as the transposed matrix of the sampling matrix Φ and automatically trained through back propagation, for a sampling rate σ = M / N, the number of rows of the measurement signal y obtained by compression sampling of the sampling module is σ × L;

[0113] The initial reconstruction process can be expressed as:

[0114]

[0115] Among them, x in is the initial reconstructed ECG signal, represents the initial reconstruction process, and y=S(·,Φ) is the compressed sampling signal.

[0116] In the Transformer-based ECG signal compression sensing method described in this embodiment, the preprocessing network includes an input layer, a two-dimensional convolutional layer, a batch normalization layer, an ELU activation layer, and an output layer connected in sequence;

[0117] The two-dimensional convolution layer is composed of three convolution kernels with a kernel size of 3*3 and a convolution layer with a step size of 1*1.

[0118] The process of learning local details of the signal through the preprocessing convolutional network can be expressed as:

[0119] x pre =x ini -C pre (x ini )

[0120] Among them, C pre (·) represents the preprocessing convolution layer, x pre Represents the signal after preprocessing convolution.

[0121] The Transformer-based ECG signal compression sensing method described in this embodiment is

[0122] The Transformer backbone network consists of a pair of encoders and decoders. The encoders and decoders are composed of modules based on the attention mechanism. The initial reconstructed signal and the preprocessed ECG signal are embedded and represented with position encoding, and then input into the encoder and decoder respectively.

[0123] The encoder is composed of multiple identical layers stacked together, each of which consists of two sublayers: a permuted multi-head self-attention layer and a feedforward network layer, where each sublayer uses residual connections and layer normalization;

[0124] The encoder process in the Transformer backbone network of the depth reconstruction module can be expressed as:

[0125] x en =T en (EM(x ini )+PE(x ini ))

[0126] Among them, T en (·) represents the encoder of Transformer, EM(·) represents the embedding layer, and init For vectorized representation, PE(·) represents the matrix function that measures the correlation of input positions;

[0127] The decoder is composed of multiple identical layers stacked together, each of which consists of three sublayers: a permuted multi-head self-attention layer, a permuted multi-head cross-attention layer, and a feedforward network layer. Residual connections and layer normalization are used in the layers.

[0128] The process of the decoder in the Transformer backbone network of the depth reconstruction module can be expressed as:

[0129] x de =T de (x en ,EM(x pre ))

[0130] Among them, T de (·) represents the decoder process of Transformer, which is similar to the encoder T en (·), but the decoder input contains the encoder output x en And the output of the embedding layer EM(x pre ), and finally, the output ECG signal data are combined to form the reconstructed ECG signal data.

[0131] In the Transformer-based ECG signal compressed sensing method described in this embodiment, each head in the rearranged multi-head self-attention layer is a scaled dot product attention mechanism combined with a depthwise separable convolution, so that each head can focus on different parts of the input sequence and calculate different weighted sums. The scaled dot product attention mechanism obtains Q, K, and V by linearly projecting the input, where K and V are rearranged by depthwise separable convolution to reduce the spatial dimension and obtain K. p , V p ; Q, K p The weights are generated through dot product operations and finally act on V p It visits any position of the signal at a constant cost and pays more attention to positions with higher attention weights, so that each head can effectively reduce complexity while focusing on different parts of the input sequence;

[0132] The scaled dot product attention mechanism is implemented by Perform linear projection to obtain Q, K, V, with dimensions d respectively. q d k and d v , the projection process is as follows:

[0133] Q,K,V=F qkv (x in );

[0134] Input signal x in After h independent linear projections, we get h groups of dimensions d q , d k and d v The Q, K, and V of ​​correspond to the input of h attention heads respectively:

[0135]

[0136] The process of rearranged multi-head self-attention operation is as follows:

[0137]

[0138] Among them, Dconv(·) represents the depth-separable convolution operation, which reduces the channel dimension of K and V to d model / 4, to achieve feature rearrangement.

[0139] The operation can be specifically expressed as:

[0140]

[0141] in, For the permuted multi-head self-attention mechanism, x in =EM(x init )+PE(x init ).

[0142] The matrix of linear projection is

[0143] The structure of the rearranged multi-head cross attention layer is exactly the same as that of the rearranged multi-head self-attention layer, so

[0144] The reordered multi-head cross attention mechanism has the same operation process as the reordered multi-head self-attention mechanism. The difference is that its scaled dot product attention mechanism is to en Perform linear projection to obtain Q, K, and calculate EM(x pre ) Perform linear projection to obtain V;

[0145] Rearrange the multi-head cross attention mechanism Expressed as:

[0146]

[0147] The Transformer-based ECG signal compressed sensing method described in this embodiment calculates the dot product of Q and all K during the projection process and divides each value by the scaling factor. To obtain weight;

[0148] The Softmax function is then applied, where the scaling factor Helps produce smoother Softmax output;

[0149] Finally, the attention weights are applied to V;

[0150] To do this, the query, keys, and values ​​are packed into a matrix

[0151] Since the self-attention mechanism can access features at any position at a constant cost, it can capture the dependencies between different heartbeat cycles in the same signal segment, thereby generating more representative features for more accurate signal reconstruction.

[0152] In the Transformer-based ECG signal compressed sensing method described in this embodiment, the final loss function constructed in S3) is specifically as follows:

[0153] The initial reconstruction loss L of the initial reconstruction module ini ,

[0154] Use x in The cosine similarity between x and x is measured as a loss function; the depth reconstruction loss L for the depth reconstruction module con ,This study uses Euclidean distance to calculate,

[0155] Therefore, the total loss L is: L = L ini +αL con

[0156] Among them, α is a weighting coefficient used to smooth the contribution differences between loss functions.

[0157] In the Transformer-based ECG signal compressed sensing method described in this embodiment, step S5) uses the percentage mean square error (PRD) and signal-to-noise ratio (SNR) quantitative indicators to verify and compare the performance of neural networks of different methods on a test set. The specific contents are as follows:

[0158] PRD is the ratio of the l2 norm of the difference between the original signal and the reconstructed signal to the l2 norm of the original signal, which describes the reconstruction error. The smaller the PRD, the better the reconstruction quality.

[0159] The formula for PRD is:

[0160]

[0161] The signal-to-noise ratio is related to the ratio of the original signal to the noise, and is also used to describe the quality of reconstruction. The larger the SNR, the better the reconstruction effect.

[0162] The SNR formula is as follows:

[0163]

[0164] Among them, x and are the original ECG signal and the reconstructed signal respectively.

[0165] It should be noted that a lightweight compressed sensing method based on Transformer is used for compressed sampling and recovery and reconstruction of ECG signals, called ECG-TransCS.

[0166] The purpose of compressed sensing of ECG signals is to further analyze and diagnose. Therefore, the requirements of clinical medicine for the quality of reconstructed signals deserve further analysis. Zigel et al. proposed using PRD to evaluate the quality of reconstructed signals and classified different PRD values ​​into different quality levels based on the signal quality perceived by clinicians. The specific correspondence is shown in Table 2. Figure 7 Based on the above research, PRD=9% is proposed as the quality standard for clinical detection signal reconstruction.

[0167] In step S4), the Transformer-based compressed sensing network is trained. The specific process is as follows:

[0168] The training set in step S1) is divided into blocks and then sent to the neural network for training; the Adam optimizer is used in the neural network training process so that the final loss function value continuously approaches the global minimum, and the polynomial learning strategy is used to dynamically adjust the learning rate;

[0169] In step S4), the Transformer-based compressed sensing network is trained. The specific process is as follows:

[0170] The training set in step S1) is divided into blocks and then sent to the neural network for training; the Adam optimizer is used in the neural network training process so that the final loss function value continuously approaches the global minimum, and the polynomial learning strategy is used to dynamically adjust the learning rate;

[0171] Specifically, the learning rate decays polynomially with the number of iterations, i.e.

[0172]

[0173] Among them, epoch represents the current training iteration number, total_epoch represents the total training iteration number, power represents the power of the polynomial, which is 0.9 in this invention. At the same time, the batch size is set to 32, and the total training iteration number is 200 rounds.

[0174] like Figure 6 The results shown here show a Transformer-based ECG signal compression sensing method, using the signal compression ratio CR to measure the compression degree of the ECG signal, and the percentage root mean square error PRD and signal-to-noise ratio SNR to measure the quality of signal reconstruction. The PRD and SNR of each record under different CRs in the test set are averaged by different methods. Figure 5The method of the present invention can still achieve good reconstruction accuracy when the ECG signal compression ratio (CR) is low (CR < 20%), while the other three algorithms perform poorly (PRD > 9%). The goal of this invention is to achieve good reconstructed signal quality (PRD < 9%) even when the signal compression ratio (CR) is less than 20%.

[0175] This demonstrates that the Transformer-based compressed sensing method for ECG signals has a strong ability to extract features from low-dimensional signals, helping to improve the performance of reconstructed signals using compressed sensing. The reconstructed signal generated by the proposed method is closer to the original signal in terms of both peak values ​​and other bands, and achieves the best performance with the lowest average PRD.

[0176] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principles of the present invention. These improvements should also be regarded as the scope of protection of the present invention.

Claims

1. A Transformer-based compressed sensing method for ECG signals, characterized by: The steps include: S1): Dataset preparation: collect and integrate the ECG data to be trained, form the ECG data into an experimental dataset, filter, de-noise and normalize the data in the experimental dataset, and then randomly divide it into a training dataset and a test dataset in a ratio of 8:2; S2): Constructing a Transformer-based compressed sensing network, wherein the Transformer-based compressed sensing network includes a learnable sampling module, a preliminary reconstruction module, and a deep reconstruction module; First, the sampling module uses a data-driven trainable perception matrix to compress and sample the initial ECG signal to obtain a low-dimensional compressed sampling signal. Simulate the process of data transmission to the server terminal; Then the signal is restored and reconstructed through two stages: preliminary reconstruction module and deep reconstruction module; S3): Construct the final loss function, which consists of two parts: initial reconstruction loss and deep reconstruction loss; S4): Train a Transformer-based compressed sensing network, i.e., divide the training dataset from step S1) into non-overlapping data segments and feed them into the neural network for training. The trained neural network is used for compressed sensing sampling and reconstruction of ECG signals. The training block size is 256×256 and the batch size is 32. The training process uses a polynomial learning strategy to adjust the learning rate. The number of training rounds is 200. Early stopping is used to avoid overfitting when the loss value calculated by the final loss function stops decreasing or oscillates. S5): Evaluation of compressed sensing reconstruction effect: cut the test data set into data blocks of the same size as the training blocks, and then send them to the neural network trained in step S4) for prediction. Then, use the percentage mean square error (PRD) and signal-to-noise ratio (SNR) quantitative indicators to verify on the test set, and compare the diagnostic performance of different methods.

2. The Transformer-based ECG signal compressed sensing method according to claim 1, characterized in that: The specific sampling method of the learnable sampling module in S2) is as follows: First, the ECG signal is divided into C×L non-overlapping data segments, where C represents the number of ECG signal channels and L represents the length of the data segment. Let Φ∈R M×N is the trainable perception matrix, M = σ × L, N = L; The sampling rate σ is as follows: σ=M / N(0≤σ; The compressed sampling process is expressed as: y = S (x, Φ) = Φ·x Where x is the original ECG signal, y is the compressed sampling ECG signal, and S(x, Φ) represents the compressed sampling operation; The sampling module jointly trains the elements of the perception matrix Φ through back-propagation from training, and the elements of the perception matrix Φ ultimately follow a Gaussian-like distribution. After joint training with the subsequent reconstruction module, the sampling module adaptively participates in the architecture to improve the signal reconstruction quality.

3. The Transformer-based ECG signal compressed sensing method according to claim 1, characterized in that: The specific process of restoring and reconstructing the signal in two stages by the preliminary reconstruction module and the deep reconstruction module in S2) is as follows: The first stage preliminary reconstruction module uses the data-driven initial reconstruction matrix To initially restore the signal by increasing the dimension, the initial reconstruction module uses the characteristics of the ECG signal learned during the training process to perform preliminary reconstruction of the signal, using the trainable initial reconstruction matrix Initial restoration signal; the initial reconstruction matrix Initialized to the transpose of the sampling matrix Φ, and automatically trained through back propagation, the elements in the matrix are trained jointly by back propagation and reconstruction modules to fully learn the prior information of the ECG signal. The signal is preliminarily reconstructed using the features of the ECG signal learned during training to obtain the initial reconstructed signal. The second-stage deep reconstruction module uses a Transformer-based hybrid reconstruction network to restore and reconstruct the signal, and uses a preprocessing convolutional network to learn the local details between ECG signals; the hybrid reconstruction network embeds the preprocessing results and the initial reconstructed signal into data representation and adds position information as the input sequence of the Transformer. The original ECG signal is used as the target sequence, and the encoder and decoder of the Transformer are used to extract the data features of the ECG signal and generate a reconstructed ECG signal; through the customized Transformer backbone network and CNN reconstruction module, the Transformer-based ECG signal compression sensing network can effectively capture the global position dependency and local details between signals. At the same time, the optimizer is used to effectively establish the mapping relationship between the original signal and the predicted signal, thereby obtaining a high-performance reconstructed ECG signal.

4. The Transformer-based ECG signal compressed sensing method according to claim 3, characterized in that: The initial reconstruction module in S2) uses the features of the ECG signal learned during the training process to perform preliminary reconstruction of the signal, using a trainable initial reconstruction matrix To restore the signal initially, Initialized as the transposed matrix of the sampling matrix Φ and automatically trained through back propagation, for a sampling rate σ = M / N, the number of rows of the measurement signal y obtained by compression sampling of the sampling module is σ × L; The initial reconstruction process can be expressed as: Among them, x init is the initial reconstructed ECG signal, represents the initial reconstruction process, and y=S(·,Φ) is the compressed sampling signal.

5. The Transformer-based ECG signal compressed sensing method according to claim 3, characterized in that: The preprocessing convolutional network includes an input layer, a two-dimensional convolutional layer, a batch normalization layer, an ELU activation layer, and an output layer connected in sequence; The process of learning local details of the signal through the preprocessing convolutional network can be expressed as: x pre =x ini -C pre (x init ) Among them, C pre (·) represents the preprocessing convolution layer, x pre Represents the signal after preprocessing convolution.

6. The Transformer-based ECG signal compressed sensing method according to claim 3, characterized in that: The Transformer backbone network consists of a pair of encoders and decoders; the initial reconstructed signal and the preprocessed ECG signal are embedded and represented, and then positionally encoded and input into the encoder-decoder. The encoder is composed of multiple identical layers stacked together, each of which consists of two sublayers: a permuted multi-head self-attention layer and a feedforward network layer, where each sublayer uses residual connections and layer normalization; The encoder process in the Transformer backbone network of the depth reconstruction module can be expressed as: x en =T en (EM(x init )+OR(x init )) Among them, T en (·) represents the encoder of Transformer, EM(·) represents the embedding layer, and init For vectorized representation, PE(·) represents the matrix function that measures the correlation of input positions; The decoder is composed of multiple identical layers stacked together, each of which consists of three sublayers: a permuted multi-head self-attention layer, a permuted multi-head cross-attention layer, and a feedforward network layer. Residual connections and layer normalization are used in the layers. The process of the decoder in the Transformer backbone network of the depth reconstruction module can be expressed as: x de =T de (x en ,EM(x pre )) Among them, T de (·) represents the decoder process of Transformer, which is similar to the encoder T en (·), but the decoder input contains the encoder output x en And the output of the embedding layer EM(x pre ), and finally, the output ECG signal data are combined to form the reconstructed ECG signal data.

7. The Transformer-based ECG signal compressed sensing method according to claim 6, characterized in that: Each head in the reordered multi-head self-attention layer is a scaled dot product attention mechanism combined with a depthwise separable convolution, so that each head can focus on different parts of the input sequence and calculate different weighted sums. The scaled dot product attention mechanism obtains Q, K, and V by linearly projecting the input, where K and V are reordered by depthwise separable convolution to reduce the spatial dimension and obtain K. p , V p ; Q, K p The weights are generated through dot product operations and finally act on V p It visits any position of the signal at a constant cost and pays more attention to positions with higher attention weights, so that each head can effectively reduce complexity while focusing on different parts of the input sequence; The scaled dot product attention mechanism is implemented by Perform linear projection to obtain Q, K, V, with dimensions d respectively. q d k and d v , the projection process is as follows: Q,K,V=F qkv (x in ); Input signal x in After h independent linear projections, we get h groups of dimensions d q , d k and d v The Q, K, and V of ​​correspond to the input of h attention heads respectively: The process of rearranged multi-head self-attention operation is as follows: Among them, Dconv(·) represents the depth-separable convolution operation, which reduces the channel dimension of K and V to d model / 4, to achieve feature rearrangement; The operation can be specifically expressed as: in, For the permuted multi-head self-attention mechanism, x in =EM(x ini )+PE(x ini ); The matrix of linear projection is The structure of the reordered multi-head cross attention layer is exactly the same as that of the reordered multi-head self-attention layer. Therefore, the reordered multi-head cross attention mechanism has the same operation process as the reordered multi-head self-attention. The difference is that its scaled dot product attention mechanism is implemented by the encoder output x en Perform linear projection to obtain Q, K, and calculate EM(x pre ) Perform linear projection to obtain V; Therefore, rearrange the multi-head cross attention mechanism Expressed as:

8. The Transformer-based ECG signal compressed sensing method according to claim 7, characterized in that: The scaled dot product attention mechanism calculates the dot product of Q and all K during the projection process and divides each value by the scale factor To obtain weight; The Softmax function is then applied, where the scaling factor Helps produce smoother Softmax output; Finally, the attention weights are applied to V; Since the self-attention mechanism can access features at any position at a constant cost, it can capture the dependencies between different heartbeat cycles in the same signal segment, thereby generating more representative features for more accurate signal reconstruction.

9. The Transformer-based ECG signal compressed sensing method according to claim 1, characterized in that: The final loss function constructed in S3) is specifically as follows: The initial reconstruction loss L of the initial reconstruction module ini , using x in The cosine similarity between x and x is measured as a loss function; the depth reconstruction loss L for the depth reconstruction module con , calculated by Euclidean distance, Therefore, the total loss L is: L = L ini +αL con Among them, α is a weighting coefficient used to smooth the contribution differences between loss functions.

10. The Transformer-based ECG signal compressed sensing method according to claim 1, characterized in that: In step S5), the percentage mean square error (PRD) and signal-to-noise ratio (SNR) quantitative indicators are used to verify and compare the performance of neural networks of different methods on the test set. The specific contents are as follows: PRD is the ratio of the l2 norm of the difference between the original signal and the reconstructed signal to the l2 norm of the original signal, which describes the reconstruction error. The smaller the PRD, the better the reconstruction quality. The formula for PRD is: The signal-to-noise ratio is related to the ratio of the original signal to the noise, and is also used to describe the quality of reconstruction. The larger the SNR, the better the reconstruction effect. The SNR formula is as follows: Among them, x and are the original ECG signal and the reconstructed signal respectively.

Images