Electrocardiosignal open set identity recognition method based on hybrid network and cross attention mechanism

Abstract

The invention discloses an electrocardiosignal open-set identity recognition method based on a hybrid network and a cross attention mechanism, and belongs to the technical field of identity recognition. The electrocardiosignal open-set identity recognition method comprises the steps that electrocardiosignals are preprocessed to obtain standardized input; constructing a hybrid network model to extract a long-time dependency relationship between multi-scale local space information and a time sequence; fusing different types of information by using a cross attention mechanism; and establishing an Openmax classification layer to realize identity tag identification in an open set environment. By means of the method, the known individual identity can be effectively recognized, unfamiliar persons are rejected, and the accuracy of electrocardiosignal identity recognition and the applicability in the practical application scene are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of identity recognition, and in particular to an open-set identity recognition method for electrocardiogram signals based on a hybrid network and a cross-attention mechanism. Background Art

[0002] Electrocardiogram (ECG), as a biometric recognition technology, has attracted widespread attention in the field of identity recognition in recent years. Compared with traditional biometrics such as face, fingerprint, iris, voiceprint, etc., ECG signals not only meet the basic characteristics required as biometrics, such as universality, uniqueness, stability and easy collection, but also have high security. As an internal physiological signal of the human body, ECG signal acquisition requires specific equipment and technology. At the same time, the liveness detection characteristics of the ECG signal itself make it difficult to be forged or copied, which makes ECG identity recognition have potential advantages in application scenarios with high security requirements.

[0003] ECG identity recognition is a typical classification problem and can be divided into closed set recognition and open set recognition. In closed set recognition, it is usually assumed that the test sample must belong to a known identity category in the training set. However, in practical applications, there is a possibility that the test sample comes from an unknown identity category, and the closed set assumption is difficult to meet the needs, so open set recognition is introduced. Compared with closed set recognition, open set recognition not only needs to correctly distinguish known identities, but also needs to be able to reject unknown identities, so it is more in line with practical application needs. Most existing ECG identity recognition systems are designed for closed set recognition, which to a certain extent limits their universality and robustness in practical applications.

[0004] In addition, the core technology of ECG identity recognition lies in the feature extraction algorithm. Traditional methods mainly rely on the time domain, frequency domain and statistical domain features of ECG signals to provide an effective basis for identity recognition. However, this method usually requires manual design of features, which is not only time-consuming and labor-intensive, but may also ignore the potential individual differences in ECG signals. In contrast, the feature extraction method based on deep learning can automatically fit the distribution of training data through multi-layer linear and nonlinear transformations to extract more discriminative features. This method avoids the limitations of manual feature extraction and improves the efficiency and accuracy of feature extraction. Nevertheless, existing neural network methods still have some limitations. For example, convolutional neural networks mainly explore the relationship between local spatial information and identity labels, while recurrent neural networks focus on capturing the long-term temporal dependence of ECG signals. This means that these methods may only utilize part of the characteristic information in the ECG signal and fail to fully and comprehensively mine the effective information of the signal.

[0005] In summary, in view of the shortcomings of closed set recognition and feature information extraction limitations in existing ECG identity recognition systems, the present invention proposes an ECG signal open set identity recognition method based on a hybrid network and a cross-attention mechanism, which aims to comprehensively extract and integrate the local spatial information and long-term dependencies of the ECG signals, and realize high-precision ECG identity recognition that is more in line with practical application needs for open set recognition scenarios. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies of the prior art and propose an open-set recognition method for ECG signals based on a hybrid network and a cross-attention mechanism. The method extracts local spatial information and temporal long-term dependencies in ECG signals through a hybrid network, and uses a cross-attention mechanism to fuse two different feature representations, while removing redundancy and fully retaining individual identity difference identification information. At the same time, by constructing an Openmax classification layer to achieve identity recognition in an open-set environment, it can not only accurately classify signals of known identity individuals, but also reject unknown identities.

[0007] To achieve the above purpose, this method provides the following technical solutions:

[0008] 1. The technical solution of the present invention includes an open-set identity recognition method for electrocardiogram signals based on a hybrid network and a cross-attention mechanism, the method comprising the following steps:

[0009] S1: Preprocess the ECG signal to obtain a standardized two-dimensional heartbeat image input;

[0010] S2: Construct a hybrid network model to extract multi-scale local spatial information and long-term dependency of time series. The input of the hybrid network model is the standardized two-dimensional heartbeat image, and the output is the multi-scale local spatial information feature representation of the standardized heartbeat image and the long-term dependency feature representation of the time series;

[0011] S3: Using a cross attention mechanism to fuse different types of information. The input of the cross attention mechanism is the output of the hybrid network model, and the output is a fusion representation of the multi-scale local spatial information features and the global long-term dependency features;

[0012] S4: Establish an Openmax classification layer to realize identity tag recognition in an open set environment. The Openmax classification model input is the output of the cross attention mechanism, and the output is the identity recognition result.

[0013] The step of preprocessing the electrocardiogram signal to obtain a standardized heartbeat image input in S1 includes:

[0014] S11: Process the input ECG signal using a high-pass filter with a cutoff frequency of 0.5 Hz, a low-pass filter with a cutoff frequency of 100 Hz, and a band-stop filter with a notch frequency range of 49.5 Hz to 50.5 Hz to obtain a clean ECG signal;

[0015] S12: using the Pan-Tompkins algorithm to locate the R and T wave apexes of the clean ECG signal, and obtaining the reference point R and T wave apex positions;

[0016] S13: based on the positioning result, the clean ECG signal is segmented with the R wave vertex as the center, the T wave vertex adjacent to the R wave vertex as the starting point, and the T wave vertex adjacent to the R wave vertex as the end point to obtain the heart beat segmentation segment;

[0017] S14: using a min-max normalization method to perform normalization processing on the obtained heart beat segmentation segments to obtain standardized heart beat segments;

[0018] S15: Use the python matplotlib toolkit to draw the standardized heartbeat segment into an RGB two-dimensional image with a size of 224*224*3, and store the RGB two-dimensional image data as a network input.

[0019] 2. According to the open set identity recognition method of ECG signal based on hybrid network and cross attention mechanism, the method comprises a hybrid network feature extraction module, a cross attention mechanism module and an Openmax classification layer. The hybrid network feature extraction module, the cross attention mechanism module and the Openmax classification layer are connected in sequence.

[0020] The hybrid network feature extraction module includes two channels: a multi-scale convolution feature extraction channel and a Transformer encoding feature extraction channel. The multi-scale convolution feature extraction channel is connected in parallel with the Transformer encoding feature extraction channel. The input is a 224*224*3 RGB two-dimensional image, and the output is a 1*768 feature vector.

[0021] 3. According to the open-set identity recognition method of ECG signals based on a hybrid network and a cross-attention mechanism, the multi-scale convolutional feature extraction channel includes a 1*1 convolutional layer, a first multi-scale convolutional feature extraction layer, a second multi-scale convolutional feature extraction layer, a third multi-scale convolutional feature extraction layer, a fourth multi-scale convolutional feature extraction layer, a fifth multi-scale convolutional feature extraction layer and a fully connected layer.

[0022] The 1*1 convolution layer, the first multi-scale convolution feature extraction layer, the second multi-scale convolution feature extraction layer, the third multi-scale convolution feature extraction layer, the fourth multi-scale convolution feature extraction layer, the fifth multi-scale convolution feature extraction layer and the fully connected layer are connected in sequence.

[0023] The convolution kernel size of the 1*1 convolution layer is 1*1, the step size is 1, the Relu activation function is adopted, and the convolution operation adopts 4 convolution kernels.

[0024] 4. According to the open-set identity recognition method of ECG signals based on a hybrid network and cross-attention mechanism, the first multi-scale convolutional feature extraction layer, the second multi-scale convolutional feature extraction layer, the third multi-scale convolutional feature extraction layer, the fourth multi-scale convolutional feature extraction layer and the fifth multi-scale convolutional feature extraction layer all include two parts: a multi-scale convolution module and a pooling module.

[0025] The multi-scale convolution module includes four branches, which are a first branch, a second branch, a third branch and a fourth branch:

[0026] The first branch is a residual connection;

[0027] The second branch is a 3*3 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same";

[0028] The third branch is a 5*5 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same";

[0029] The third branch is a 7*7 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same";

[0030] Each convolution operation of the multi-scale convolution module in the first multi-scale convolution feature extraction layer adopts 4 convolution kernels;

[0031] Each convolution operation of the multi-scale convolution module in the second multi-scale convolution feature extraction layer uses 16 convolution kernels;

[0032] Each convolution operation of the multi-scale convolution module in the third multi-scale convolution feature extraction layer uses 32 convolution kernels;

[0033] Each convolution operation of the multi-scale convolution module in the fourth multi-scale convolution feature extraction layer uses 64 convolution kernels;

[0034] Each convolution operation of the multi-scale convolution module in the fifth multi-scale convolution feature extraction layer uses 128 convolution kernels.

[0035] The pooling modules all use maximum pooling, the pooling layer size is 2*2, and the step size is 2.

[0036] 5. According to the open-set identity recognition method of ECG signals based on hybrid network and cross-attention mechanism, the fully connected layer is used to expand the output of the fifth multi-scale convolutional feature extraction layer and map it to a 1*768 feature vector, using a Relu activation function. The fully connected layer input is the expanded feature vector of the output of the fifth multi-scale convolutional feature extraction layer, and the output is a 1*768 feature vector, and the output feature vector dimension is the same as the output feature dimension of the Transformer encoding feature extraction channel.

[0037] 6. According to the open-set identity recognition method of ECG signals based on a hybrid network and a cross-attention mechanism, the Transformer encoding feature extraction channel includes 5 stacked Transformer encoders.

[0038] The construction of the hybrid network model in step 2 to extract the long-term dependency of the time series is completed by the Transformer encoding feature extraction channel. The input is a 224*224*3 RGB two-dimensional heartbeat image. The feature extraction process includes the following steps:

[0039] S21: Linearly project the two-dimensional heartbeat image using a convolution operation with a size of 16*16, a step size of 16, and a channel number of 768. The dimension after linear projection is 196*768, that is, 196 vector sequences are obtained after projection, and the dimension of each sequence is 768;

[0040] S22: Add a classification mark with a dimension of 1*768 to the front end of the vector sequence. After being concatenated with the 196*768 vector sequence, the dimension is 197*768.

[0041] S23: Generate position code using cosine function, superimpose it to the vector sequence with dimension of 197*768, and use the result as input of Transformer encoder;

[0042] S24: Use the five stacked Transformer encoders to perform feature extraction on the input 197*768 vector sequence, wherein each Transformer encoder includes a multi-head attention module and a multi-layer perceptron module. The number of attention heads in the multi-head attention module is set to 8, and the multi-layer perceptron includes two fully connected layers, and the number of nodes is 768->3072->768.

[0043] S25: Finally, the classification label obtained after extraction mapping is used as the feature extracted by the Transformer encoding feature extraction channel, and the dimension is 1*768, which is the same as the output feature dimension of the multi-scale convolution feature extraction channel.

[0044] 7. According to the open set identity recognition method of ECG signal based on hybrid network and cross attention mechanism, the input of the cross attention mechanism module is the 1*768 feature vector extracted by the multi-scale convolution feature extraction channel and the Transformer encoding feature extraction channel, respectively denoted as X Conv , X Trans .

[0045] The process of fusing different types of information using the cross-attention mechanism in step 3 includes the following steps:

[0046] S31: Generate the X by linear transformation Conv , X Trans Corresponding query, key and value. In the linear transformation process, the linear transformation matrices corresponding to the query, key and value are recorded as W Q , W K , W V , the linear transformation matrix is ​​learned by the back propagation algorithm. Conv The corresponding query, key, and value are Q Conv , K Conv V Conv , X Trans The corresponding query, key, and value are Q Trans , K Trans , V Trans ;

[0047] S32: Calculate the attention score according to the query, key, and value, X Conv , X Trans The corresponding attention score is:

[0048]

[0049] Where Softmax(·) is the Softmax function, After being mapped by the Softmax function, the Attention1 value is X Conv Corresponding to the attention score, After being mapped by the Softmax function, the Attention2 value is X Trans Corresponding attention score.

[0050] S33: Calculate the fused feature vector according to the attention score:

[0051] X Fuse =Attention1*V Conv +Attention2* VTrans

[0052] Where X Fuse is the fused feature vector.

[0053] 8. According to the open set identity recognition method of ECG signal based on hybrid network and cross attention mechanism, the Openmax classification layer is a three-layer fully connected network, which includes input layer, hidden layer and output layer. The input layer is the output of the cross attention mechanism module, which contains 768 nodes, the hidden layer contains 384 nodes, and the number of nodes in the output layer is related to the number of individual categories of identity recognition. If the number of individual categories of identity recognition is N Total , then the number of output layer nodes is N Total In step 4, establishing the Openmax classification layer to realize identity tag recognition in an open set environment includes the following steps:

[0054] S41: Openmax classification layer training:

[0055] S42: open set recognition of the Openmax classification layer.

[0056] 9. According to the open set identity recognition method of ECG signal based on hybrid network and cross attention mechanism, the Openmax classification layer training in step 41 includes the following steps:

[0057] S411: Network weight and bias optimization: According to the requirements of the ten-fold cross-validation experimental method, the data set is divided into a training set and a test set, of which the training set accounts for nine parts and the test set accounts for one part. The back propagation algorithm is used to optimize the parameters of the entire network to obtain the weight and bias of the Openmax classification layer. During the optimization process, the number of output nodes is N Total At this time, the network is a closed set recognition, which only faces the samples of individuals with known identities in the training set. The Openmax classification layer is equivalent to the Softmax classifier.

[0058] S412: Weibull distribution parameter fitting: After the training is completed, for the training set samples, the Openmax classification layer hidden layer output set S corresponding to the correctly classified training set data is taken, and the obtained hidden layer output set S is divided into N according to the category. Total data subsets, each subset is denoted as S j For each subset S j , take the mean, and get N Total Activation vector MAV j , the activation vector expression is calculated as follows:

[0059]

[0060] Among them, N j is the number of samples of category j, S ij is the i-th sample in the data subset corresponding to category j. Calculate the sample S in each subset ijto their corresponding activation vector MAV j The Euclidean distance of each subset is fitted with Weibull distribution using the built-in function wblfit of Matlab software to obtain the scale parameter τ of j Weibull distributions. j , shape parameter λ j With the location parameter κ j .

[0061] The network weights, biases and Weibull distribution parameters included in the Openmax classification layer together constitute the model parameters of the Openmax classification layer.

[0062] 10. According to the ECG signal open set identity recognition method based on hybrid network and cross attention mechanism, the Openmax classification open set recognition in step S42 includes the following steps:

[0063] S421: Output score correction: For each test sample Sample, obtain the output score S of the sample in the hidden layer of the Openmax classification layer through network mapping. Sample , the output score v of the output layer Sample . In descending order, Sample The internal scores are sorted, and the sorted sequence is recorded as v Sample The correction formula for the i-th score value in the medium probability sequence is as follows:

[0064]

[0065] Among them, ω i To correct the weight, the calculation formula is as follows:

[0066]

[0067] Where dis(·) is the Euclidean distance, MAV i is the activation vector corresponding to category i, τ i , i With κ i is the Weibull distribution parameter corresponding to category i.

[0068] S422: Assign pseudo scores to the unfamiliar and unknown category: Assign label 0 to the unfamiliar and unknown identity. The pseudo score of label 0 is as follows:

[0069]

[0070] S423: Classification probability calculation under open set conditions: Using Softmax function to classify the sequence Normalize and obtain the open set recognition condition, and identify the test sample Sample as category 0~N Total The probability sequence

[0071] S424: Open set identification: take the maximum value p in the probability sequence max , if the maximum value p max is p0 or p max If the maximum value p is less than the preset threshold ε, it is judged as an unknown identity and rejected; max Not p0 and p max If it is greater than the preset threshold ε, it is determined to be a known identity, and the category label corresponding to the maximum value is assigned to the test sample.

[0072] The beneficial effects of the present invention are as follows: constructing a dual-channel feature extraction hybrid network including a multi-scale convolution feature extraction channel and a Transformer encoding feature extraction channel, which can obtain the long-term dependency feature representation of the ECG sequence while extracting the multi-scale local space representation, and fully and comprehensively mine the information related to identity recognition in the electrocardiogram signal; establishing a cross-attention mechanism module, and enhancing the model's perception of key features by realizing the effective fusion of the two channel features, improving the discriminability of the feature representation, and improving the recognition accuracy; utilizing the Openmax classification layer to assign pseudo scores to unfamiliar and unknown identity labels based on the output of the Softmax classifier through output score correction, and obtaining the classification probability of the sample in an open set environment, thereby realizing open set recognition, and making the recognition result more in line with the actual application environment requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The present invention is further described below with reference to the accompanying drawings and embodiments.

[0074] Figure 1 Shown is a flow chart of an open-set ECG identity recognition method based on a hybrid network and a cross-attention mechanism according to an embodiment of the present invention;

[0075] Figure 2 Shown is a pre-processing flow chart according to an embodiment of the present invention;

[0076] Figure 3 It is a structural diagram of an open-set ECG identity recognition method based on a hybrid network and a cross-attention mechanism according to an embodiment of the present invention;

[0077] Figure 4 The figure shows a structure diagram of a hybrid network feature extraction module according to an embodiment of the present invention.

[0078] Figure 5 The figure shows a multi-scale convolutional feature extraction layer structure according to an embodiment of the present invention.

[0079] Figure 6 The figure is a flow chart of Transformer encoding feature extraction channel feature extraction according to an embodiment of the present invention.

[0080] Figure 7 The cross-attention mechanism module structure diagram according to an embodiment of the present invention is shown as follows

[0081] Figure 8 The figure shows a flow chart of obtaining the parameters of the Weibull distribution of the Openmax classification layer according to an embodiment of the present invention.

[0082] Fig. 9 The figure shows the flow chart of the open set recognition of the Openmax classification layer according to the embodiment of the present invention.

[0083] Fig.10 The figure is a flow chart of the result determination of ECG open-collection identity recognition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0084] This section will describe the specific embodiments of the present invention in detail in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

[0085] In the description of the present invention, “several” means one or more, “more” means more than two, “greater than”, “less than”, “exceed”, etc. are understood as not including the number itself, and “above”, “below”, “within”, etc. are understood as including the number itself.

[0086] In the description of the present invention, the consecutive numbering of the method steps is for the convenience of review and understanding. Combined with the overall technical scheme of the present invention and the logical relationship between the various steps, adjusting the implementation order between the steps will not affect the technical effect achieved by the technical scheme of the present invention.

[0087] In the description of the present invention, unless otherwise clearly defined, words such as setting should be understood in a broad sense, and technicians in the technical field can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0088] In this embodiment, the ECG data adopts the PTB public data set and the ECG-ID public data set. In the implementation process, the PTB data set is used as a known identity individual data set to carry out a closed set recognition effect test, and the ECG-ID data set is used as an unknown individual data set to carry out an open set recognition effect test.

[0089] like Figure 1As shown, this embodiment provides an open-set identity recognition method for electrocardiogram signals based on a hybrid network and a cross-attention mechanism, comprising the following steps:

[0090] S1: Preprocess the ECG signal to obtain a standardized two-dimensional heartbeat image input;

[0091] S2: Construct a hybrid network model to extract multi-scale local spatial information and long-term dependencies of time series. The input of the hybrid network model is a standardized two-dimensional heartbeat image, and the output is a multi-scale local spatial information feature representation of the standardized heartbeat image and a feature representation of the long-term dependencies of the time series;

[0092] S3: Use the cross attention mechanism to fuse different types of information. The input of the cross attention mechanism is the output of the hybrid network model, and the output is the fusion representation of multi-scale local spatial information features and global long-term dependency relationship features;

[0093] S4: Establish an Openmax classification layer to realize identity tag recognition in an open set environment. The input of the Openmax classification model is the output of the cross attention mechanism, and the output is the identity recognition result.

[0094] like Figure 2 As shown, the steps of preprocessing all ECG signals to obtain standardized heartbeat image input include:

[0095] S11: Process the input ECG signal using a high-pass filter with a cutoff frequency of 0.5 Hz, a low-pass filter with a cutoff frequency of 100 Hz, and a band-stop filter with a notch frequency range of 49.5 Hz to 50.5 Hz to obtain a clean ECG signal;

[0096] S12: Use the Pan-Tompkins algorithm to locate the R and T wave apexes of the clean ECG signal to obtain the reference point R and T wave apex positions;

[0097] S13: based on the positioning result, the clean ECG signal is segmented with the R wave vertex as the center, the T wave vertex adjacent to the R wave vertex as the starting point, and the T wave vertex adjacent to the R wave vertex as the end point to obtain the heart beat segmentation segment;

[0098] S14: using a min-max normalization method to normalize the obtained heart beat segmentation segments to obtain standardized heart beat segments;

[0099] S15: Use the python matplotlib toolkit to plot the standardized heart beat segments as an RGB two-dimensional image with a size of 224*224*3, and store the RGB two-dimensional image data as the network input.

[0100] like Figure 3As shown, the ECG signal open set identity recognition method based on hybrid network and cross attention mechanism includes a hybrid network feature extraction module, a cross attention mechanism module and an Openmax classification layer. The hybrid network feature extraction module, the cross attention mechanism module and the Openmax classification layer are connected in sequence.

[0101] like Figure 4 As shown, the hybrid network feature extraction module includes two channels: a multi-scale convolution feature extraction channel and a Transformer encoding feature extraction channel. The multi-scale convolution feature extraction channel is connected in parallel with the Transformer encoding feature extraction channel. The input is a 224*224*3 RGB two-dimensional image, and the output is a 1*768 feature vector.

[0102] like Figure 4 As shown, the multi-scale convolutional feature extraction channel includes a 1*1 convolutional layer, a first multi-scale convolutional feature extraction layer, a second multi-scale convolutional feature extraction layer, a third multi-scale convolutional feature extraction layer, a fourth multi-scale convolutional feature extraction layer, a fifth multi-scale convolutional feature extraction layer and a fully connected layer.

[0103] The 1*1 convolution layer, the first multi-scale convolution feature extraction layer, the second multi-scale convolution feature extraction layer, the third multi-scale convolution feature extraction layer, the fourth multi-scale convolution feature extraction layer, the fifth multi-scale convolution feature extraction layer and the fully connected layer are connected in sequence.

[0104] The convolution kernel size of the 1*1 convolution layer is 1*1, the step size is 1, the Relu activation function is used, and the convolution operation uses 4 convolution kernels.

[0105] like Figure 5 As shown, the first multi-scale convolutional feature extraction layer, the second multi-scale convolutional feature extraction layer, the third multi-scale convolutional feature extraction layer, the fourth multi-scale convolutional feature extraction layer and the fifth multi-scale convolutional feature extraction layer all include two parts: a multi-scale convolution module and a pooling module.

[0106] The multi-scale convolution module consists of four branches, namely the first branch, the second branch, the third branch and the fourth branch:

[0107] The first branch is a residual connection;

[0108] The second branch is a 3*3 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same";

[0109] The third branch is a 5*5 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same";

[0110] The third branch is a 7*7 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same";

[0111] Each convolution operation of the multi-scale convolution module in the first multi-scale convolution feature extraction layer uses 4 convolution kernels;

[0112] Each convolution operation of the multi-scale convolution module in the second multi-scale convolution feature extraction layer uses 16 convolution kernels;

[0113] Each convolution operation of the multi-scale convolution module in the third multi-scale convolution feature extraction layer uses 32 convolution kernels;

[0114] Each convolution operation of the multi-scale convolution module in the fourth multi-scale convolution feature extraction layer uses 64 convolution kernels;

[0115] Each convolution operation of the multi-scale convolution module in the fifth multi-scale convolution feature extraction layer uses 128 convolution kernels.

[0116] The pooling modules all use maximum pooling, the pooling layer size is 2*2, and the step size is 2.

[0117] The fully connected layer is used to expand the output of the fifth multi-scale convolutional feature extraction layer and map it to a 1*768 feature vector, using the Relu activation function. The fully connected layer input is the expanded feature vector of the output of the fifth multi-scale convolutional feature extraction layer, and the output is a 1*768 feature vector. The dimension of the output feature vector is the same as the output feature dimension of the Transformer encoding feature extraction channel.

[0118] like Figure 6 As shown, the Transformer encoding feature extraction channel contains 5 stacked Transformer encoders.

[0119] In step 2, the hybrid network model is constructed to extract the long-term dependency of the time series, which is completed by the Transformer encoding feature extraction channel. The input is a 224*224*3 RGB two-dimensional heartbeat image. The feature extraction process includes the following steps:

[0120] S21: Linearly project the two-dimensional heartbeat image using a convolution operation with a size of 16*16, a step size of 16, and a channel number of 768. The dimension after linear projection is 196*768, that is, 196 vector sequences are obtained after projection, and the dimension of each sequence is 768;

[0121] S22: Add a classification tag with a dimension of 1*768 to the front of the vector sequence. After concatenating it with the 196*768 vector sequence, the dimension is 197*768.

[0122] S23: Generate positional encoding using the cosine function, superimpose it on a projection vector sequence with a dimension of 197*768, and use the result as the Transformer encoder input;

[0123] S24: Use 5 stacked Transformer encoders to extract features from the input 197*768 vector sequence. Each Transformer encoder contains a multi-head attention module and a multi-layer perceptron module. The number of attention heads in the multi-head attention module is set to 8, and the multi-layer perceptron contains two fully connected layers with a node number of 768->3072->768.

[0124] In the embodiment, only the first Transformer encoder is subjected to linear projection, classification tags are added, and position coding is superimposed. The following four Transformer encoders do not need to repeat the above operations, and directly use the 197*768-dimensional vector sequence output by the previous layer as input.

[0125] S25: Finally, the classification label obtained after extraction mapping is used as the feature extracted by the Transformer encoding feature extraction channel, and the dimension is 1*768, which is the same as the output feature dimension of the multi-scale convolution feature extraction channel.

[0126] like Figure 7 As shown in the figure, the input of the cross attention mechanism module is the 1*768 feature vector extracted by the multi-scale convolution feature extraction channel and the Transformer encoding feature extraction channel, which are respectively denoted as X Conv , X Trans .

[0127] The process of fusing different types of information using the cross-attention mechanism in step 3 includes the following steps:

[0128] S31: Generate the X by linear transformation Conv , X Trans Corresponding query, key and value. In the linear transformation process, the linear transformation matrices corresponding to the query, key and value are recorded as W Q , W K , W V , the linear transformation matrix is ​​learned by the back propagation algorithm. Connv The corresponding query, key, and value are Q Conv , K Conv V Conv , X Trans The corresponding query, key, and value are Q Trans , K Trans , V Trans ;

[0129] S32: Calculate attention score based on query, key, and value, X Conv , X Trans The corresponding attention score is:

[0130]

[0131] Where Softmax(·) is the Softmax function, After being mapped by the Softmax function, the Attention1 value is X Conv Corresponding to the attention score, After being mapped by the Softmax function, the Attention2 value is X Trans Corresponding attention score.

[0132] S33: Calculate the fused feature vector based on the attention score:

[0133] X Fuse =Attention1*V conv +Attention2*V Trans

[0134] Where X Fuse is the fused feature vector.

[0135] 9. The Openmax classification layer is a three-layer fully connected network, which consists of an input layer, a hidden layer, and an output layer. The input layer is the output of the cross-attention mechanism module and contains 768 nodes. The hidden layer contains 384 nodes. The number of nodes in the output layer is related to the number of identity recognition individual categories. If the number of identity recognition individual categories is N Total , then the number of output layer nodes is N Total In the embodiment, since the number of individuals included in PTB is selected to be 290, N Total is 290.

[0136] In step 4, the Openmax classification layer is established to realize identity label recognition in an open set environment, which includes the following steps:

[0137] S41: Openmax classification layer training;

[0138] S42: open set recognition of the Openmax classification layer.

[0139] 10. The Openmax classification layer training in step 41 includes the following steps:

[0140] S411: Network weight and bias optimization: According to the requirements of the ten-fold cross-validation experimental method, the PTB data set is divided into a training set and a test set, of which the training set accounts for nine parts and the test set accounts for one part. The back propagation algorithm is used to optimize the parameters of the entire network to obtain the weight and bias of the Openmax classification layer. During the optimization process, the number of output nodes is N Total At this time, the network is a closed set recognition, which only faces the samples of individuals with known identities in the training set. The Openmax classification layer is equivalent to the Softmax classifier.

[0141] In the embodiment, the data used in the training process are all from the PTB dataset, and the signal data of 290 individuals in the PTB dataset need to be covered. The training process is mainly oriented to closed set recognition.

[0142] S412: Figure 8 As shown, Weibull distribution parameter fitting: After the training is completed, for the training set samples in PTB, the Openmax classification layer hidden layer output set S corresponding to the correctly classified training set data is taken, and the obtained hidden layer output set S is divided into N according to the category Total data subsets, each subset is denoted as S j For each subset S j , take the mean, and get N Total Activation vector MAV j , the activation vector expression is calculated as follows:

[0143]

[0144] Among them, N j is the number of samples of category j, S ij is the i-th sample in the data subset corresponding to category j. Calculate the sample S in each subset ij to their corresponding activation vector MAV j The Euclidean distance of each subset is fitted with Weibull distribution using the built-in function wblfit of Matlab software to obtain the scale parameter τ of j Weibull distributions. j , shape parameter λ j With the location parameter κ j .

[0145] The network weights, biases and Weibull distribution parameters contained in the Openmax classification layer together constitute the model parameters of the Openmax classification layer.

[0146] like Fig. 9 As shown, the Openmax classification open set recognition in step S42 includes the following steps:

[0147] S421: Output score correction: For each test sample Sample in the ECG-ID dataset, the output score S of the sample in the hidden layer of the Openmax classification layer is obtained through network mapping. Sample , the output score v of the output layer Sample . In descending order, Sampple The internal scores are sorted, and the sorted sequence is recorded as v Sample The correction formula for the i-th score value in the medium probability sequence is as follows:

[0148]

[0149] Among them, ω i To correct the weight, the calculation formula is as follows:

[0150]

[0151] Where dis(·) is the Euclidean distance, MAV i is the activation vector corresponding to category i, τ i , i With κ i is the Weibull distribution parameter corresponding to category i.

[0152] S422: Assign pseudo scores to the unfamiliar and unknown category: Assign label 0 to the unfamiliar and unknown identity. The pseudo score of label 0 is as follows:

[0153]

[0154] S423: Classification probability calculation under open set conditions: Using Softmax function to classify the sequence Normalize and obtain the open set recognition condition, and identify the test sample Sample as category 0~N Total The probability sequence

[0155] S424: Fig.10 As shown, open set recognition: take the maximum value p in the probability sequence max , if the maximum value p max is p0 or p max If the maximum value p is less than the preset threshold ε, it is judged as an unknown identity and rejected; max Not p0 and p max If it is greater than the preset threshold ε, it is determined to be a known identity, and the category label corresponding to the maximum value is assigned to the test sample. Here the threshold ε is 0.9.

Claims

1. An open-set identity recognition method for ECG signals based on a hybrid network and cross-attention mechanism. It is characterized in that The method comprises the following steps: S1: Preprocess the ECG signal to obtain a standardized two-dimensional heartbeat image input; S2: constructing a hybrid network model to extract multi-scale local spatial information and long-term dependency of time series, wherein the input of the hybrid network model is the standardized two-dimensional heartbeat image, and the output is the multi-scale local spatial information feature representation of the standardized heartbeat image and the long-term dependency feature representation of the time series; S3: using a cross attention mechanism to fuse different types of information, wherein the input of the cross attention mechanism is the output of the hybrid network model, and the output is a fusion representation of the multi-scale local spatial information features and the long-term dependency relationship features of the time series; S4: Establish an Openmax classification layer to realize identity label recognition in an open set environment. The input of the Openmax classification model is the output of the cross attention mechanism, and the output is the identity recognition result. The step of preprocessing the electrocardiogram signal to obtain a standardized heartbeat image input in S1 includes: S11: Process the input ECG signal using a high-pass filter with a cutoff frequency of 0.5 Hz, a low-pass filter with a cutoff frequency of 100 Hz, and a band-stop filter with a notch frequency range of 49.5 Hz to 50.5 Hz to obtain a clean ECG signal; S12: using the Pan-Tompkins algorithm to locate the R and T wave apexes of the clean ECG signal, and obtaining the reference point R and T wave apex positions; S13: based on the positioning result, the clean ECG signal is segmented with the R wave vertex as the center, the T wave vertex adjacent to the R wave vertex as the starting point, and the T wave vertex adjacent to the R wave vertex as the end point to obtain the heart beat segmentation segment; S14: using a min-max normalization method to perform normalization processing on the obtained heart beat segmentation segments to obtain standardized heart beat segments; S15: Use the python matplotlib toolkit to draw the standardized heartbeat segment into an RGB two-dimensional image with a size of 224*224*3, and store the RGB two-dimensional image data as a network input.

2. According to claim 1, an open set identity recognition method of electrocardiogram signals based on a hybrid network and a cross attention mechanism, It is characterized in that The method comprises a hybrid network feature extraction module, a cross attention mechanism module and an Openmax classification layer. The hybrid network feature extraction module, the cross attention mechanism module and the Openmax classification layer are connected in sequence. The hybrid network feature extraction module includes two channels: a multi-scale convolution feature extraction channel and a Transformer encoding feature extraction channel. The multi-scale convolution feature extraction channel is connected in parallel with the Transformer encoding feature extraction channel. The input is a 224*224*3 RGB two-dimensional image, and the output is a 1*768 feature vector.

3. According to claim 2, an open set identity recognition method of electrocardiogram signals based on a hybrid network and a cross attention mechanism, It is characterized in that The multi-scale convolution feature extraction channel includes a 1*1 convolution layer, a first multi-scale convolution feature extraction layer, a second multi-scale convolution feature extraction layer, a third multi-scale convolution feature extraction layer, a fourth multi-scale convolution feature extraction layer, a fifth multi-scale convolution feature extraction layer and a fully connected layer; The 1*1 convolution layer, the first multi-scale convolution feature extraction layer, the second multi-scale convolution feature extraction layer, the third multi-scale convolution feature extraction layer, the fourth multi-scale convolution feature extraction layer, the fifth multi-scale convolution feature extraction layer and the fully connected layer are connected in sequence; The convolution kernel size of the 1*1 convolution layer is 1*1, the step size is 1, the Relu activation function is adopted, and the convolution operation adopts 4 convolution kernels.

4. According to claim 3, an open set identity recognition method of electrocardiogram signals based on a hybrid network and a cross attention mechanism, It is characterized in that The first multi-scale convolution feature extraction layer, the second multi-scale convolution feature extraction layer, the third multi-scale convolution feature extraction layer, the fourth multi-scale convolution feature extraction layer and the fifth multi-scale convolution feature extraction layer each comprise two parts: a multi-scale convolution module and a pooling module; The multi-scale convolution module includes four branches, which are respectively a first branch, a second branch, a third branch and a fourth branch: The first branch is a residual connection; The second branch is a 3*3 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same"; The third branch is a 5*5 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same"; The third branch is a 7*7 convolution kernel with a step size of 1, a Relu activation function, and Padding is set to "same"; Each convolution operation of the multi-scale convolution module in the first multi-scale convolution feature extraction layer adopts 4 convolution kernels; Each convolution operation of the multi-scale convolution module in the second multi-scale convolution feature extraction layer uses 16 convolution kernels; Each convolution operation of the multi-scale convolution module in the third multi-scale convolution feature extraction layer uses 32 convolution kernels; Each convolution operation of the multi-scale convolution module in the fourth multi-scale convolution feature extraction layer uses 64 convolution kernels; Each convolution operation of the multi-scale convolution module in the fifth multi-scale convolution feature extraction layer uses 128 convolution kernels; The pooling modules all use maximum pooling, the pooling layer size is 2*2, and the step size is 2.

5. According to claim 3, an open set identity recognition method of electrocardiogram signals based on a hybrid network and a cross attention mechanism, It is characterized in that The fully connected layer is used to expand the output of the fifth multi-scale convolutional feature extraction layer and map it into a 1*768 feature vector, using a Relu activation function; The fully connected layer input is the expanded feature vector output by the fifth multi-scale convolutional feature extraction layer, and the output is a 1*768 feature vector, and the output feature vector dimension is the same as the output feature dimension of the Transformer encoding feature extraction channel.

6. According to claim 2, an open set identity recognition method of electrocardiogram signals based on a hybrid network and a cross attention mechanism, It is characterized in that The Transformer encoding feature extraction channel includes 5 stacked Transformer encoders; The construction of the hybrid network model in step 2 to extract the long-term dependency of the time series is completed by the Transformer encoding feature extraction channel. The input is a 224*224*3 RGB two-dimensional heartbeat image. The feature extraction process includes the following steps: S21: Linearly project the two-dimensional heartbeat image using a convolution operation with a size of 16*16, a step size of 16, and a channel number of 768. The dimension after linear projection is 196*768, that is, 196 vector sequences are obtained after projection, and the dimension of each sequence is 768; S22: Add a classification mark with a dimension of 1*768 to the front end of the vector sequence. After being concatenated with the 196*768 vector sequence, the dimension is 197*768. S23: Generate a position code using a cosine function, superimpose it on the projection vector sequence with a dimension of 197*768, and use the result as the input of the Transformer encoder; S24: Use the 5 stacked Transfomer encoders to perform feature extraction on the input 197*768 vector sequence, each of the Transfomer encoders includes a multi-head attention module and a multi-layer perceptron module, wherein the number of attention heads of the multi-head attention module is set to 8, and the multi-layer perceptron includes two fully connected layers, and the number of nodes is 768->3072->768. Only the first Transformer encoder performs linear projection, adds classification tags and superimposed position encoding. The next four Transformer encoders do not need to repeat the above operations and directly use the 197*768-dimensional vector sequence output by the previous layer as input; S25: Finally, the classification label obtained after extraction mapping is used as the feature extracted by the Transformer encoding feature extraction channel, and the dimension is 1*768, which is the same as the output feature dimension of the multi-scale convolution feature extraction channel.

7. According to claim 2, an open set identity recognition method of electrocardiogram signals based on a hybrid network and a cross attention mechanism, It is characterized in that The input of the cross attention mechanism module is the 1*768 feature vector extracted by the multi-scale convolution feature extraction channel and the Transformer encoding feature extraction channel, respectively denoted as X Conv , X Trans ; The process of fusing different types of information using the cross-attention mechanism in step 3 includes the following steps: S31: Generate the X by linear transformation Conv , X Trans Corresponding query, key and value. In the linear transformation process, the linear transformation matrices corresponding to the query, key and value are recorded as W Q , W K , W V , the linear transformation matrix is ​​learned by the back propagation algorithm, X Conv The corresponding query, key, and value are Q Conv , K Conv V Conv , X Trans The corresponding query, key, and value are Q Trans , K Trans , V Trans ; S32: Calculate the attention score according to the query, key, and value, X Conv , X Trans The corresponding attention score is: Where Softmax(·) is the Softmax function, After mapping by Softmax function, we get Attention 1 Value is X Conv Corresponding to the attention score, After mapping by Softmax function, we get Attention 2 Value is X Trans Corresponding attention score; S33: Calculate the fused feature vector according to the attention score: X Fuse =Attention 1 *V Conv +Attention 2 *V Trans Where X Fuse is the fused feature vector.

8. According to claim 2, an open set identity recognition method of electrocardiogram signals based on a hybrid network and a cross attention mechanism, It is characterized in that The Openmax classification layer is a three-layer fully connected network, which includes an input layer, a hidden layer and an output layer. The input layer is the output of the cross attention mechanism module and contains 768 nodes. The hidden layer contains 384 nodes. The number of nodes in the output layer is related to the number of identity recognition individual categories. If the number of identity recognition individual categories is N Total , then the number of output layer nodes is N Total In step 4, establishing the Openmax classification layer to realize identity tag recognition in an open set environment includes the following steps: S41: Openmax classification layer training; S42: the Openmax classification layer performs open set recognition.

9. According to claim 8, an open set identity recognition method of electrocardiogram signals based on a hybrid network and a cross attention mechanism, It is characterized in that The Openmax classification layer training described in step 41 includes the following steps: S411: Optimizing network weights and biases: According to the requirements of the ten-fold cross-validation experimental method, the data set is divided into a training set and a test set, of which the training set accounts for nine parts and the test set accounts for one part; the back propagation algorithm is used to optimize the parameters of the entire network to obtain the weights and biases of the Openmax classification layer; during the optimization process, the number of output nodes is N Total , at this time, the network is a closed set recognition, which only faces the samples of individuals with known identities in the training set, and the Openmax classification layer is equivalent to the Softmax classifier; S412: Weibull distribution parameter fitting: After the training is completed, for the training set samples, the Openmax classification layer hidden layer output set S corresponding to the correctly classified training set data is taken, and the obtained hidden layer output set S is divided into N according to the category. Total data subsets, each subset is denoted as S j For each subset S j , take the mean, and get N Total Activation vector MAV j , the activation vector expression is calculated as follows: Among them, N j is the number of samples of category j, S ij is the i-th sample in the data subset corresponding to category j. Calculate the sample S in each subset ij to their corresponding activation vector MAV j The Euclidean distance of each subset is fitted with Weibull distribution using the built-in function wblfit of Matlab software to obtain the scale parameter τ of j Weibull distributions. j , shape parameter λ j With the location parameter κ j The network weights, biases and Weibull distribution parameters contained in the Openmax classification layer together constitute the model parameters of the Openmax classification layer.

10. The open set identity recognition method of electrocardiogram signal based on hybrid network and cross attention mechanism according to claim 8, It is characterized in that The Openmax classification open set recognition in step S42 includes the following steps: S421: Output score correction: For each test sample Sample, obtain the output score S of the sample in the hidden layer of the Openmax classification layer through network mapping. Sample , the output score v of the output layer Sample , in descending order, Sample The internal scores are sorted, and the sorted sequence is recorded as v Sample The correction formula for the i-th score value in the medium probability sequence is as follows: Among them, ω i To correct the weight, the calculation formula is as follows: Where dis(·) is the Euclidean distance, MAV i is the activation vector corresponding to category i, τ i , i With κ i is the Weibull distribution parameter corresponding to category i; S422: Assign pseudo scores to the unfamiliar and unknown category: Assign label 0 to the unfamiliar and unknown identity. The pseudo score of label 0 is as follows: S423: Classification probability calculation under open set conditions: Using Softmax function to classify the sequence Normalize and obtain the open set recognition condition, and identify the test sample Sample as category 0~N Total The probability sequence S424: Open set identification: take the maximum value p in the probability sequence max , if the maximum value p max For p 0 or p max If the maximum value p is less than the preset threshold ε, it is judged as an unknown identity and rejected; max Not for p 0 And p max If it is greater than the preset threshold ε, it is determined to be a known identity, and the category label corresponding to the maximum value is assigned to the test sample.

Images