Electrocardiosignal filtering identification method based on CLIP model
Through the ECG signal filtering identification method based on the CLIP model, combined with the self-attention mechanism and the cross-modal encoder, the problem that traditional methods are difficult to suppress multiple noises and high computing complexity is solved, and efficient and accurate ECG signal filtering is achieved, which is suitable for portable devices.
Patent Information
- Application Number
- CN202510492652.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2025-08-01
AI Technical Summary
Traditional electrocardiogram filtering identification methods are difficult to suppress multiple noises at the same time, especially when processing nonlinear and non-stationary signals, the effect is limited, and the deep learning-based method has high computational complexity and high resource consumption, which limits the application of portable electrocardiogram monitoring equipment.
The ECG signal filtering recognition method based on the CLIP model is used to train the neural network through the training set and the verification set, and combine the Glamic angle field, Markov transition field and recursive graph to transform the ECG sequence into RGB images. The mixed features are extracted using the self-attention mechanism and a cross-modal encoder to achieve high-efficiency filtering of the ECG signal.
It improves the accuracy and robustness of ECG signal recognition, reduces the computational complexity and power consumption, and realizes low-power portable ECG signal filtering recognition.
Smart Images
Figure CN120392016A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of signal processing, and particularly relates to an electrocardiogram signal filtering and recognition method based on the CLIP model. Background Art
[0002] Electrocardiogram is an important tool for studying heart diseases, which can comprehensively reflect the heart conduction system and its own pathological conditions, and is one of the main means for clinical diagnosis of cardiovascular diseases. In recent years, with the rapid development of electronic information technology and mobile Internet, the concept of mobile health has gradually emerged. The popularization and application of intelligent wearable devices provide a convenient way for the monitoring of common heart diseases, and can continuously monitor the heart health status of individuals in real time. Among them, dynamic electrocardiogram monitoring plays an important role in arrhythmia detection. By recording electrocardiogram signals for a long time, the occurrence and frequency of arrhythmia can be detected in time, which is of great significance for clinical diagnosis.
[0003] Traditional electrocardiogram signal filtering and recognition methods (such as low-pass, high-pass, band-stop filtering, wavelet transform, adaptive filtering, etc.) are mainly designed for specific types of noise (such as power frequency interference, baseline drift or electromyographic noise), but it is difficult to suppress multiple noises simultaneously, and the effect is limited when dealing with non-linear and non-stationary signals. In addition, these methods may lose important features of electrocardiogram signals, such as the sharpness of QRS complex or the detailed changes of ST segment.
[0004] Although the electrocardiogram signal filtering and recognition algorithms based on deep learning perform well in signal processing accuracy and noise suppression ability, their high computational complexity and resource consumption have become important factors restricting practical applications. Such algorithms usually rely on powerful hardware support, resulting in a significant increase in power consumption, which not only poses a challenge to the low-power requirements of embedded devices, but also increases the hardware cost and energy consumption. In addition, the model complexity and harsh requirements of the operating environment also limit their wide application in portable or real-time electrocardiogram monitoring devices.
[0005] In contrast, the self-attention mechanism can capture the global dependence and multi-scale features of electrocardiogram signals, take into account the temporal information and the learning of complex patterns, and has stronger generalization ability and adaptability, providing a new idea for efficient and accurate electrocardiogram signal filtering. Summary of the Invention
[0006] The purpose of the present invention is to overcome the deficiencies of the prior art, and provide an electrocardiogram signal filtering and recognition method based on the CLIP model, aiming at special non-stationary periodic signals such as electrocardiogram, to solve the problems that traditional electrocardiogram measuring instruments are difficult to suppress multiple noises simultaneously, have limited effect when dealing with non-linear and non-stationary signals, are difficult to take into account temporal information and have poor generalization effect, and the traditional deep learning-based filtering algorithms have high requirements for hardware and power consumption.
[0007] The technical solution adopted by the present invention is as follows:
[0008] An electrocardiogram signal filtering and recognition method based on the CLIP model is as follows:
[0009] S1 Train and validate the neural network through the training set and the validation set, which are obtained by dividing the data set; during training and validation, each segment of the electrocardiogram sequence in the training set and the validation set is input into the neural network in turn.
[0010] The process of modeling the neural network is as follows:
[0011] (1) Transform the electrocardiogram sequence into three grayscale images through the Gramian angular field, Markov transition field, and recurrence plot respectively, and then splice the three grayscale images as the R, G, and B channels into an RGB image I.
[0012] (2) Calculate the Mel cepstral coefficient features and second-order spectral features of the electrocardiogram sequence, and then calculate the first-order difference and second-order difference of the Mel cepstral coefficient features. Splice the Mel cepstral coefficient features and the two difference features to obtain the Mel cepstral coefficient feature array S′ k , k = 1, 2, …, t, and reconstruct to obtain the sequence S′ = [S′1, S′2, …, S′ k , …, S′ t , where t is the sequence length.
[0013] (3) Build two one-dimensional convolutional layers and a GELU activation function layer. The two one-dimensional convolutional layers perform feature mapping on the sequence S′ in step (2) in turn, and then the GELU activation function layer uses an adaptive gating mechanism to achieve feature extraction to obtain the sequence S.
[0014] (4) Obtain the image feature and sequence feature of the RGB image I obtained in step (1) and the sequence S obtained in step (3) through the CLIP model, calculate the cosine similarity between the global feature of the image and the global feature of the sequence, and use the cosine similarity as the CL loss to optimize the CLIP model.
[0015] (5) Use the image feature and sequence feature as the input of the cross-modal encoder, and calculate the hybrid feature F of the image and the sequence using multi-head cross-attention.
[0016] (6) Pass the hybrid feature F in step (5) through a 1×1 convolutional layer, a fully connected layer, and a softmax function to obtain the electrocardiogram type output.
[0017] S2 Resample the real-time collected human electrocardiogram signal at the same acquisition frequency as the data set, and then divide the resampled electrocardiogram signal into several segments of electrocardiogram sequence X = [x1, x2, …, x m, where m is the number of sampling points in each ECG sequence. If the last ECG sequence has less than m sampling points, it is padded with zeros. Each ECG sequence is sequentially input into the neural network to obtain the output of the ECG type corresponding to each ECG sequence.
[0018] Preferably, the CLIP model includes a visual encoder and a text encoder.
[0019] More preferably, the visual encoder uses ViT (Vision Transformer). After dividing the RGB image into N patch blocks, it is mapped using a fully connected layer to obtain patch embeddings. The patch embeddings are added with position encoding as the input to the encoder block of the transformer. After passing through several encoder blocks, image features are obtained.
[0020] More preferably, the ViT processing process is as follows: Given an RGB image with size (H, W, C), where H is the image height, W is the image width, and C is the number of channels. The RGB image is divided into patch blocks and tiled, and the matrix size is (N, C). Then the result is mapped through a fully connected layer to obtain patch embeddings, and the matrix size of the patch embeddings is (N, d m ). After adding the global feature CLS, the matrix size is (N + 1, d m ). Then, after adding the position encoding E pos , it passes through several encoder blocks of the transformer to obtain image features with a size still (N + 1, d m ).
[0021] The calculation formula for obtaining the input of the encoder block of the transformer in ViT is as follows:
[0022]
[0023] Among them, z0 is the input of the encoder block of the transformer after position encoding, x cls is the vector representation of the global feature CLS, is the i-th patch block, E is the fully connected layer (linear mapping layer) that maps the patch block to a d m -dimensional encoded vector, E pos is the learnable (adaptive) position encoding, R is the real number field matrix space, and P is the size of the patch block.
[0024] More preferably, the text encoder is also based on the encoder block of the transformer and is composed of a plurality of cascaded encoder blocks; denote the sequence S = [S1, S2, …, S k , …, S L T , where any S k ∈S is a d m -dimensional feature vector. Then the matrix size of the sequence S is (L, d m ). After adding the global feature CLS to the sequence S, the matrix size becomes (L + 1, d m ). Then, the result after position encoding is input into a plurality of encoder blocks of the transformer to obtain sequence features, and the size is still (L + 1, d m ).
[0025] More preferably, the step (5) is specifically as follows: <s
[0026] Suppose the matrix size of the image features is (N1, d m ), and the matrix size of the sequence features is (N2, d m ). The number of heads of the multi-head cross-attention is n, where N1 = N + 1 and N2 = L + 1. In the multi-head cross-attention, first, the image features and the sequence features pass through n fully connected layers with a matrix size of . Each fully connected layer obtains the vector K and the vector V in the attention mechanism according to the image features, and obtains the vector Q in the attention mechanism according to the sequence features. Among them, the vector K and the vector Then, use the formula to calculate the attention score matrix, with a size of N2×N1, where d k is a dimension of the vector K Subsequently, apply the Softmax function to convert the attention scores into a probability distribution, and then calculate the similarity through the formula . Concatenate the n Attention with a matrix size of obtained through n fully connected layers to obtain an output feature with a matrix size of . Finally, input the output feature into N encoder blocks in sequence to obtain the mixed features, and the size of the mixed features is (N2, d m ).
[0027] More preferably, in the step (6), a 1×1 convolutional layer reduces the mixed features F with a size of (N2, d m ) to reduced-dimensional features with a size of (1, d m ), and then passes through a layer with a size of (d m , the fully connected layer of (1, class_num) obtains the output features of size (1, class_num), where class_num is the number of ECG types. The values in the output features of the fully connected layer are converted into probabilities using the softmax function, and the value with the maximum probability is taken as the final ECG type output.
[0028] More preferably, the encoder block structure of the transformer is as follows:
[0029] Perform multi-head self-attention (MSA) calculation on the input Z, then add the output of the multi-head self-attention and the original input to form a residual connection, and perform layer normalization on the result. The result Z L ′ passes through a multi-layer perceptron (MLP), and then adds it to the result of layer normalization to form a residual connection. Finally, perform layer normalization on the output of the residual connection again to obtain the final output Z L . The expression is as follows:
[0030] Z′ L =LN(MSA(Z)+Z)
[0031] Z L =LN(MLP(Z′ L )+Z′ L )
[0032] More preferably, the multi-layer perceptron (MLP) includes two linear transformations (fully connected layers), and a non-linear activation function ReLU is applied in the middle of the two linear transformations.
[0033] The beneficial effects of the present invention are as follows:
[0034] 1. The present invention uses the CLIP model to learn hybrid features, improves the accuracy of ECG signal recognition, and increases the robustness of recognition. It solves the problems that traditional filtering recognition methods are difficult to suppress multiple noises simultaneously, and have limited effects when dealing with non-linear and non-stationary signals, and lose important features. Among them, the self-attention mechanism can capture the global dependence and multi-scale features of ECG signals, take into account the learning of temporal information and complex patterns, and has stronger generalization ability and adaptability, providing a new idea for efficient and accurate filtering recognition of ECG signals.
[0035] 2. The present invention solves the problems of high computational complexity and resource consumption of traditional deep learning methods, and realizes low-power and portable deployment. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is the flowchart of the present invention.
[0037] Figure 2This is the structural block diagram of the encoder block in the present invention. Specific embodiments
[0038] The present invention will be further described below with reference to the accompanying drawings.
[0039] As Figure 1 shown, the electrocardiogram signal filtering and recognition method based on the CLIP model includes the following steps:
[0040] S1 Train and validate the neural network through the training set and the validation set, and the training set and the validation set are obtained by dividing the data set; during training and validation, each segment of the electrocardiogram sequence in the training set and the validation set is input into the neural network in turn. In this embodiment, the open-source MIT-BIH data set is used as the data set.
[0041] Among them, the neural network modeling process is as follows:
[0042] (1) Respectively convert the electrocardiogram sequence into three grayscale images through the Gramian angular field, the Markov transition field, and the recurrence plot, and then splice the three grayscale images as the R, G, and B channels into an RGB image I. The conversion through the Gramian angular field, the Markov transition field, and the recurrence plot fully converts the information of the time series into image information, preparing for the subsequent feature fusion of the image and the sequence.
[0043] (2) Calculate the Mel cepstral coefficient features and the second-order spectrum features of the electrocardiogram sequence, and then calculate the first-order difference and the second-order difference of the Mel cepstral coefficient features. Splice the Mel cepstral coefficient features and the two difference features to obtain the Mel cepstral coefficient feature array S′ k , k = 1, 2, …, t, and reconstruct to obtain the sequence S′ = [S′1, S′2, …, S′ k , …, S′ t , where t is the sequence length. Specifically, the Mel cepstral coefficient features can distinguish spectral details and reflect electromyographic noise or high-frequency pathological features, but there is loss of time-domain information, thus ignoring the phase information, while the time-domain waveform of the electrocardiogram signal (such as the QRS width) is crucial. Therefore, the present invention combines it with the Gramian angular field, the Markov transition field, and the recurrence plot that consider the time-domain information to achieve better results.
[0044] (3) Build two one-dimensional convolutional layers and a GELU activation function layer. The two one-dimensional convolutional layers perform feature mapping on the sequence S′ in step (2) in turn, and then the GELU activation function layer uses an adaptive gating mechanism to achieve feature extraction to obtain a sequence S that meets the input requirements of the text encoder in the CLIP model. Among them, the GELU activation function layer is a one-dimensional convolutional layer using the GELU function (Gaussian error linear unit activation function).
[0045] (4) Obtain the RGB image I obtained in step (1) and the sequence S obtained in step (3) through the CLIP model to obtain image features and sequence features, calculate the cosine similarity between the global feature of the image and the global feature of the sequence, and use the cosine similarity as the CL loss to optimize the CLIP model. Among them, the CLIP model includes a visual encoder and a text encoder.
[0046] The visual encoder uses ViT (Vision Transformer). After dividing the RGB image into N patch blocks, it is mapped using a fully connected layer to obtain patch embeddings. The patch embeddings are added with positional encodings as the input to the encoder block of the transformer. A CLS encoding similar to the bert model is used in ViT to enable it to extract the global feature of the image.
[0047] Since the transformer structure was initially used in the NLP field, applying it to the CV field has the problem of the quadratic computational cost of images, greatly increasing the computational cost. Therefore, in the visual encoder of the present invention, ViT is used to break up the RGB image into N patch blocks, reducing the computational cost. Given an RGB image with size (H, W, C), where H is the image height, W is the image width, and C is the number of channels (3 for RGB images in this case), in this implementation, preprocessing (such as resizing) is performed in advance to make H and W of the RGB image equal. In this embodiment, the patch block is set to 16*16. At this time After the RGB image is divided into patch blocks and tiled, the matrix size is (N, C). Then, the result is mapped through a fully connected layer to obtain patch embeddings, and the matrix size of the obtained patch embeddings is (N, d m ). After adding the global feature CLS, the matrix size becomes (N + 1, d m ). Then, the positional encoding E pos is added, and after passing through several encoder blocks of the transformer, image features are obtained, and the size is still (N + 1, d m ).
[0048] Among them, ViT adopts the form of absolute positional encoding, uses the positional encoding E pos to represent the absolute position of each patch block, and in a learnable manner, adds the positional encoding E pos to the network parameters and continuously updates them during the network training process. The calculation formula for obtaining the input of the encoder block of the transformer in ViT is as follows:
[0049]
[0050] Among them, z0 is the input of the encoder block of the transformer after position encoding, and x cls is the vector representation of the global feature CLS. is the i-th patch block, E is a fully connected layer (linear mapping layer) used to map the patch block to d m dimensional encoded vector, d m is determined according to the size of the CLIP model. Generally, a larger dimension brings higher accuracy but introduces greater computational cost. In the case of ECG detection where real-time requirements for time series are strong, the computing power of the actual hardware needs to be considered. In this embodiment, d m = 768, E pos is a learnable (adaptive) position encoding, R is the real number field matrix space, and P is the size of the patch block.
[0051] The text encoder (denoted as the first bert module) is also based on the encoder block of the transformer and is composed of a cascade of several encoder blocks. Since the sequence S that meets the input requirements of the text encoder has been obtained in step (3), let S = [S1, S2,..., S k ,..., S L T , where any S k ∈S is a d m dimensional feature vector. Then the matrix size of the sequence S is (L, d m ). After adding the global feature CLS to the sequence S, the matrix size becomes (L + 1, d m ). Then, the result after position encoding is input into several encoder blocks of the transformer to obtain sequence features, and the size is still (L + 1, d m ).
[0052] Since the dimensions of the global feature CLS of the sequence and the global feature CLS of the image are the same, the cosine similarity of the two global features can be calculated, and the calculation result is used as the value of the CL loss. The purpose of this loss is to minimize the cosine similarity of the global features, so that the image features and sequence features passing through the single encoder are in a similar latent space.
[0053] (5) Use the image features and sequence features as the input of the cross-modal encoder (denoted as the second bert module), and calculate the mixed feature F of the image and the sequence using multi-head cross-attention, as follows:
[0054] Let the matrix size of the image features be (N1, d m ), the matrix size of the sequence features be (N2, d m ), and the number of heads of the multi-head cross-attention be n, where N1 = N + 1, N2 = L + 1; in the multi-head cross-attention, first, the image features and the sequence features pass through n fully connected layers with a matrix size of . Each fully connected layer obtains the vector K (abbreviation of Key, which is the key vector in the self-attention mechanism) and the vector V (abbreviation of Value, which is the value vector in the self-attention mechanism) in the attention mechanism according to the image features, and obtains the vector Q (abbreviation of Query, which is the query vector in the self-attention mechanism) in the attention mechanism according to the sequence features. Among them, the vector K and the vector vector Then, use the formula to calculate the attention score matrix (i.e., the weight distribution matrix, with a size of N2×N1), where d k is a dimension of the vector K Subsequently, apply the Softmax function to convert the attention scores into a probability distribution, and then calculate the similarity through the formula . To obtain multi-level feature information, concatenate the n matrices with a size of obtained through n fully connected layers to get an output feature with a matrix size of ; finally, input this output feature into N encoder blocks in sequence to obtain a mixed feature, and the size of the mixed feature is (N2, d m ).
[0055] (6) Pass the mixed feature F in step (5) through a 1×1 convolutional layer, a fully connected layer, and the softmax function to obtain the electrocardiogram type output. Specifically, the 1×1 convolutional layer reduces the mixed feature F with a size of (N2, d m ) to a reduced-dimensional feature with a size of (1, d m ), and then passes it through a fully connected layer with a size of (d m , class_num) to obtain an output feature with a size of (1, class_num), where class_num is the number of electrocardiogram types. Use the softmax function to convert the values in the output feature of the fully connected layer into probabilities, and take the value with the maximum probability as the final electrocardiogram type output.
[0056] S2 re-samples the real-time collected human electrocardiogram signals at the same acquisition frequency as the dataset (360Hz in this embodiment) to ensure the consistency of the electrocardiogram data after re-sampling with the dataset acquisition frequency. Then, divide the re-sampled electrocardiogram signals into several segments of electrocardiogram sequences X = [x1, x2,..., xm , where m is the number of sampling points in each segment of the electrocardiogram sequence. If the last segment of the electrocardiogram sequence has fewer than m sampling points, it is padded with zeros. Each segment of the electrocardiogram sequence is sequentially input into the neural network to obtain the electrocardiogram type output corresponding to each segment of the electrocardiogram sequence.
[0057] The present invention respectively extracts unimodal image features and sequence features based on the CLIP model, and then uses a cross-modal encoder for modal fusion. Among them, both the CLIP model and the cross-modal encoder are implemented based on the encoder block of the transformer, as Figure 2 shown. The basic structure of the encoder block of the transformer is as follows:
[0058] Perform multi-head self-attention (MSA) calculation on the input Z. The dimension of the result obtained by the MSA calculation is the same as that of the input; add the output of the multi-head self-attention and the original input to form a residual connection, and perform layer normalization on the result. The result Z L ' passes through a layer of multi-layer perceptron (MLP), and then adds it to the result of layer normalization to form a residual connection. Finally, perform layer normalization on the output of the residual connection again to obtain the final output Z L . The expression is as follows:
[0059] Z′ L =LN(MSA(Z)+Z)
[0060] Z L =LN(MLP(Z′ L )+Z′ L )
[0061] The present invention cascades multiple encoder blocks, that is, the output of the upper-layer encoder block is used as the input of the lower-layer encoder block. The final outputs of the CLIP model and the cross-modal encoder are of the same dimension as the input.
[0062] Among them, layer normalization is to transform the input into a distribution with a mean of 0 and a variance of 1 to alleviate the problems of gradient disappearance and explosion.
[0063] The multi-layer perceptron (MLP) includes two linear transformations (fully connected layers), and a non-linear activation function ReLU is applied in the middle of the two linear transformations. The mathematical expression of the non-linear activation function ReLU is:
[0064] ReLU(x)=max(0,x)
[0065] Among them, ReLU(x) represents the output of the ReLU activation function; x represents the input of the ReLU activation function; max(0,x) represents taking the larger value between the input and zero.
[0066] The multi-layer perceptron (MLP) is expressed as follows:
[0067] (1) The input is mapped through the first linear transformation (fully connected layer), and the formula for the first linear transformation is:
[0068] z1 = hW1 + b1
[0069] where z1 is the output after the first linear transformation, W1 is the weight of the first linear transformation, h is the output of the previous layer, and b1 is the bias of the first linear transformation.
[0070] (2) The non-linear unit activation function (ReLU) is applied to the output of the first linear transformation to perform a non-linear transformation on each dimension, setting all negative values to 0. The formula is as follows:
[0071] a1 = ReLU(z1)
[0072] where a1 is the output of the non-linear unit activation function.
[0073] (3) The output of the non-linear unit activation function is mapped through a second linear transformation back to the same dimension as the input to interface with subsequent network layers. The formula for the second linear transformation is:
[0074] z2 = a1W2 + b2
[0075] where z2 is the output after the first linear transformation, W2 is the weight of the second linear transformation, and b2 is the bias of the second linear transformation.
Claims
1. An electrocardiogram signal filtering and recognition method based on the CLIP model, characterized in that: The method is as follows: S1 trains and validates the neural network through the training set and the validation set, which are obtained by dividing the data set; during training and validation, each segment of the electrocardiogram sequence in the training set and the validation set is sequentially input into the neural network; The neural network modeling process is as follows: (1) The electrocardiogram sequence is respectively transformed into three grayscale images through the Gramian Angular Field, the Markov Transition Field, and the recurrence plot, and then the three grayscale images are respectively used as the R, G, and B channels to be stitched into an RGB image I; (2) Calculate the Mel cepstral coefficient features and second-order spectrum features of the electrocardiogram sequence, and then calculate the first-order difference and second-order difference of the Mel cepstral coefficient features; splice the Mel cepstral coefficient features and the two difference features to obtain the Mel cepstral coefficient feature array S ′ k , k = 1, 2, …, t, and reconstruct to obtain the sequence S′ = [S′1, S′2, …, S′ k , …, S′ t , t is the sequence length; (3) Build two one-dimensional convolutional layers and a GELU activation function layer. The two one-dimensional convolutional layers perform feature mapping on the sequence S' in step (2) in sequence, and then the GELU activation function layer uses an adaptive gating mechanism to achieve feature extraction to obtain the sequence S; (4) The RGB image I obtained in step (1) and the sequence S obtained in step (3) are used to obtain image features and sequence features through the CLIP model. Calculate the cosine similarity between the global feature of the image and the global feature of the sequence, and use the cosine similarity as the CL loss to optimize the CLIP model; (5) Use the image features and sequence features as the input of the cross-modal encoder, and calculate the mixed feature F of the image and the sequence using multi-head cross-attention; (6) Pass the mixed feature F in step (5) through a 1×1 convolutional layer, a fully connected layer, and a softmax function to obtain the electrocardiogram type output; S2 resamples the real-time collected human electrocardiogram signals using the same acquisition frequency as the dataset, and then divides the resampled electrocardiogram signals into several segments of electrocardiogram sequences X = [x1, x2, …, x m , where m is the number of sampling points in each segment of the electrocardiogram sequence. If the last segment of the electrocardiogram sequence has less than m sampling points, it is padded with zeros. Each segment of the electrocardiogram sequence is input into the neural network in turn to obtain the output of the electrocardiogram type corresponding to each segment of the electrocardiogram sequence.
2. The electrocardiogram signal filtering and recognition method based on the CLIP model according to claim 1, wherein: The CLIP model includes a visual encoder and a text encoder.
3. The electrocardiogram signal filtering and recognition method based on the CLIP model according to claim 2, wherein: The visual encoder uses ViT. After dividing the RGB image into N patch blocks, it is mapped using a fully connected layer to obtain image patch embeddings. The image patch embeddings plus the position encoding are used as the input of the encoder block of the transformer. After several encoder blocks, image features are obtained.
4. The electrocardiogram signal filtering and recognition method based on the CLIP model according to claim 3, characterized in that: The specific ViT processing process is as follows: Given an RGB image with a size of (H, W, C), where H is the image height, W is the image width, and C is the number of channels, the RGB image is patched and tiled, and the resulting matrix size is (N, C). Then, the result is mapped through a fully connected layer to obtain patch embeddings, and the matrix size of the patch embeddings is (N, d m ). After adding the global feature CLS, the matrix size becomes (N + 1, d m ). Then, the position encoding E pos is added, and after passing through several encoder blocks of the transformer, image features are obtained, and the size remains (N + 1, d m ); The calculation formula for obtaining the input of the encoder block of the transformer in ViT is as follows: Among them, z0 is the input of the encoder block of the transformer after position encoding, and x cls is the vector representation of the global feature CLS, is the i-th patch block, E is a fully connected layer that maps the patch block to a d m dimensional encoded vector, E pos is a learnable position encoding, R is the real number field matrix space, and P is the size of the patch block.
5. The electrocardiogram signal filtering and recognition method based on the CLIP model according to claim 4, characterized in that: The text encoder is also based on the encoder block of the transformer and is composed of several cascaded encoder blocks; denote the sequence S = [S1, S2, …, S k , …, S L T , where any S k ∈ S is a d m -dimensional feature vector, then the matrix size of the sequence S is (L, d m ). After adding the global feature CLS to the sequence S, the matrix size becomes (L + 1, d m ). Then, the result after position encoding is input into several encoder blocks of the transformer to obtain the sequence feature, and the size is still (L + 1, d m ). 6. The electrocardiogram signal filtering and recognition method based on the CLIP model according to claim 5, characterized in that: The specific content of step (5) is as follows: Let the matrix size of the image features be (N1, d m ), and the matrix size of the sequence features be (N2, d m ). The number of heads of the multi-head cross-attention is n. Here, N1 = N + 1, N2 = L + 1. In the multi-head cross-attention, the image features and the sequence features are first passed through n fully connected layers with a matrix size of . Each fully connected layer obtains the vector K and the vector V in the attention mechanism according to the image features, and obtains the vector Q in the attention mechanism according to the sequence features. Among them, the vector K and the vector Then, use the formula to calculate the attention score matrix, with a size of N2×N1, where d k is a dimension of the vector K Subsequently, apply the Softmax function to convert the attention scores into a probability distribution, and then calculate the similarity through the formula Concatenate the n Attention matrices with a matrix size of obtained through n fully connected layers to get an output feature with a matrix size of . Finally, input this output feature into the encoder blocks of N transformers in sequence to obtain a mixed feature, and the size of the mixed feature is (N2, d m ).
7. The electrocardiogram signal filtering and recognition method based on the CLIP model according to claim 6, wherein: The specific step (6) is that a 1×1 convolutional layer reduces the size of the mixed feature F of (N2, d m ) to the reduced feature of (1, d m ). Then, through a fully connected layer of size (d m , class_num), the output feature of size (1, class_num) is obtained. class_num is the number of electrocardiogram types. The values in the output feature of the fully connected layer are converted into probabilities by using the softmax function, and the value with the maximum probability is taken as the final electrocardiogram type output.
8. The electrocardiogram signal filtering and recognition method based on the CLIP model according to any one of claims 3 to 6, characterized in that: The structure of the encoder block of the transformer is as follows: Perform multi-head self-attention calculation on the input Z, then add the output of the multi-head self-attention to the original input to form a residual connection, and perform layer normalization on the result; the result Z of layer normalization L ′ Pass through a layer of multi-layer perceptron, then add it to the result of layer normalization to form a residual connection, and finally perform layer normalization on the output of the residual connection again to obtain the final output Z L ; The expression is as follows: Z′ L = LN(MSA(Z) + Z) Z L = LN(MLP(Z' L ) + Z' L )。 9. The electrocardiogram signal filtering and recognition method based on the CLIP model according to claim 8, characterized in that: The multi-layer perceptron includes two linear transformations, and a non-linear activation function ReLU is applied in the middle of the two linear transformations.