Clustering method of ECG data based on nonparametric spherical surface variation auto encoder

Abstract

To provide an efficient, exact, and flexible clustering method of ECG data based on nonparametric spherical surface variation auto encoder by cluster analysis of electrocardiographic data by coupling advantages of deep learning and a nonparametric Bayesian model with each other.SOLUTION: In a method for improving identification exactness and predictive ability to heart diseases to cluster end-to-end electrocardiographic data based on a deep variation auto encoder, a framework of a nonparametric Bayesian model based on a Pitman-Yor process mixed model is adopted to construct an infinite mixed model based on von Mises-Fisher (VMF) probability distribution.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention belongs to the fields of machine learning and medical data analysis, and particularly relates to the processing and analysis of electrocardiogram (ECG) data. Specifically, it relates to a method for clustering ECG data based on a non-parametric spherical variational autoencoder.

Background Art

[0002] In the prior art, ECG data analysis generally relies on conventional machine learning methods and basic statistical models, such as simple classification algorithms, time series analysis, and basic clustering techniques. However, these methods face challenges in processing large-scale or complex ECG data, especially in accurately identifying and classifying various heart conditions. In contrast, the Variational Auto-Encoder (VAE), as an advanced deep generative model, has shown excellent data generation and feature extraction capabilities in the fields of image and audio processing, but its application to ECG data analysis remains relatively limited.

[0003] Therefore, the method for clustering ECG data based on the non-parametric spherical variational autoencoder according to the present invention aims to effectively handle complex and large-scale ECG data analysis tasks.

Summary of the Invention

[0004] To achieve the above object, the present invention creates and adopts a technical solution that combines the advantages of deep learning and non-parametric Bayesian models to provide a more efficient, accurate, and flexible solution for the cluster analysis of ECG data.

[0005] Furthermore, this method is an end-to-end electrocardiogram data clustering method based on a deep variational autoencoder, which adopts the framework of a nonparametric Bayesian model based on the Pitman-Yor process mixture model to construct an infinite mixture model based on the VMF probability distribution. The method includes step 1) of collecting an ECG dataset, and step 2) of modeling an ECG data model using a nonparametric spherical variational autoencoder.

[0006] Furthermore, the VMF mixture model includes a probability decoder, a deep generative model, and a deep generative model based on a hyperspherical latent space, all of which follow the VMF probability distribution.

[0007] The beneficial effects after improvement by this technical solution are to improve the accuracy and efficiency of clustering by combining the strong feature extraction ability of the variational autoencoder and the flexibility of the nonparametric model. The model architecture of the variational autoencoder is constructed by a nonparametric Bayesian method based on the Pitman-Yor process mixture model, enabling the model to automatically adjust the complexity of its latent space and dynamically adjust the number of clusters according to the characteristics of the data itself. According to the VMF probability distribution, a clustering mechanism more effective for specific data types is provided compared to the conventional Gaussian distribution.

[0008] Furthermore, in step 1) of collecting an ECG dataset, JPEG2025091403000002.jpg527 is used as the dataset containing the collected N ECG data, and each data JPEG2025091403000003.jpg44 in it is a D-dimensional ECG data vector.

[0009] Furthermore, in step 2) of modeling an ECG data model using a nonparametric spherical variational autoencoder, it is a nonparametric mixture model composed of an infinite number of VMF distributions defined as follows, JPEG2025091403000004.jpg627; wherein, JPEG2025091403000005.jpg717, JPEG2025091403000006.jpg46 is the mixing coefficient of cluster JPEG2025091403000007.jpg43, and JPEG2025091403000008.jpg512 and JPEG2025091403000009.jpg719 satisfy the conditions.

[0010] Furthermore, the prior probability of the mixing coefficient JPEG2025091403000010.jpg45 is constructed based on the Pitman-Yor process mixture model using the Stick-Breaking representation method. In this mixture model, the mixing coefficient JPEG2025091403000011.jpg45 is shown as follows, JPEG2025091403000012.jpg1131; wherein, JPEG2025091403000013.jpg620 follows the Beta distribution, and its expression form is as follows, JPEG2025091403000014.jpg875; wherein, JPEG2025091403000015.jpg59 is the Beta distribution, JPEG2025091403000016.jpg54 is the discount parameter in the Pitman-Yor process mixture model, and JPEG2025091403000017.jpg517 satisfies the conditions, JPEG2025091403000018.jpg54 is the density parameter, and JPEG2025091403000019.jpg515 satisfies the conditions.

[0011] Furthermore, the non-parametric Bayesian model is a non-parametric VMF mixture model in which the latent space follows a prior, When JPEG2025091403000020.jpg43 is selected, The latent representation of JPEG2025091403000021.jpg44 JPEG2025091403000022.jpg53 is obtained through the following process by sampling from its latent distribution, JPEG2025091403000023.jpg1381; where, JPEG2025091403000024.jpg611, JPEG2025091403000025.jpg612, JPEG2025091403000026.jpg617 are VMF distributions, JPEG2025091403000027.jpg54 and JPEG2025091403000028.jpg44 are respectively the position parameter and concentration parameter of the 43rd VMF probability distribution in the non-parametric VMF mixture model, and the function JPEG2025091403000030.jpg512 is the modified 49th order Bessel function of the first kind.

[0012] Furthermore, a probability decoder JPEG2025091403000032.jpg614 generates a sample JPEG2025091403000033.jpg44 as shown by the following equation, JPEG2025091403000034.jpg638; where the parameters JPEG2025091403000035.jpg56 and JPEG2025091403000036.jpg44 is obtained automatically by training a neural network where the input is JPEG2025091403000037.jpg53 and the parameter is JPEG2025091403000038.jpg45. JPEG2025091403000039.jpg513.

[0013] Furthermore, the deep generative model is defined as follows: JPEG2025091403000040.jpg6128; where the objective function of the deep generative model is JPEG2025091403000041.jpg513's variational lower bound and is obtained by the following formula: JPEG2025091403000042.jpg17128; where JPEG2025091403000043.jpg58 represents the expected value calculation, JPEG2025091403000044.jpg620 is the true posterior distribution JPEG2025091403000045.jpg520 is the variational posterior distribution that approximates it.

[0014] Furthermore, the objective function of the deep generative model based on the hypersphere latent space is redefined as follows: JPEG2025091403000046.jpg21128; where JPEG2025091403000047.jpg652 is the KL divergence between the variational posterior distribution JPEG2025091403000048.jpg620 and JPEG2025091403000049.jpg616.

[0015] Furthermore, the infinite mixture model based on the VMF probability distribution is JPEG2025091403000050.jpg613's variational posterior distribution Expected value for JPEG2025091403000051.jpg612 By calculating JPEG2025091403000052.jpg628, assign the data to the cluster with the maximum probability Assign JPEG2025091403000053.jpg44 to complete the clustering

[0016] The beneficial effects of the present invention are as follows 1. End-to-end ECG data clustering method: The present invention proposes an end-to-end electrocardiogram data clustering method based on a deep variational autoencoder. By utilizing deep learning technology, this method realizes the direct conversion from raw data input to clustering result output, greatly simplifying the data processing flow 2. Application of nonparametric Bayesian mixture model: The proposed method adopts a nonparametric Bayesian mixture model as the prior distribution of the latent space. One of the important advantages of such a model is that it can automatically adjust the number of clusters, avoiding the limitation of manual setting of the number of clusters in conventional clustering methods. This characteristic makes the model more flexible and enables it to handle various datasets of different scales and complexities 3. Improvement of clustering performance by adopting the VMF probability distribution: The present invention uses the VMF probability distribution as the basic distribution in the nonparametric mixture model instead of the conventional Gaussian distribution. Since the VMF distribution is particularly suitable for processing data distributed on the high-dimensional sphere, it shows better clustering performance when processing normalized high-dimensional data such as electrocardiogram data. In addition, the VMF distribution also helps to reduce the overfitting problem and improve the generalization ability of the model

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0018] Hereinafter, in order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail with reference to the accompanying drawings. However, this part of the description is merely illustrative and explanatory, and does not limit the protection scope of the present invention in any way.

[0019] The clustering method for electrocardiogram (ECG) data based on the nonparametric spherical variational autoencoder proposed by the present invention has been verified for its effectiveness on the disclosed ECG5000 dataset. This dataset contains 5000 time-series data with a length of 140 each representing one heartbeat cycle. The dataset is divided into a total of five categories: one normal class (N) and four abnormal classes (R-on-T premature ventricular contraction (R-on-T PVC), premature ventricular contraction (PVC), supraventricular premature contraction or ectopic beat (SP or EB), and unclassified beat (UB)). The raw dataset is divided into a training set and a test set, and each set contains data from all categories. It is noted that there is an obvious imbalance in the numbers of these categories, with a large number of normal data. Since the model of the present invention adopts unsupervised learning and does not rely on tagged data, the training set and the test set are analyzed here as a whole.

[0020] At the same time, the present invention uses the Windows 10 operating system, the Python programming language, and the PyTorch machine learning library. The clustering effect is measured by the clustering accuracy (ACC). Here, the model is trained using the Adam optimizer, with a learning rate of 0.003 and a dropout rate of 0.2. The training process includes 200 epochs, and the batch size is set to 32 to perform gradient calculation and weight update. The dimension of the latent space is set to 5 dimensions.

[0021] As shown in FIGS. 1-4 in the specific embodiment, the method for clustering ECG data based on a non-parametric spherical variational autoencoder adopts the framework of a non-parametric Bayesian model (FIG. 2) based on the Pitman-Yor process mixture model to construct an infinite mixture model (FIG. 3) based on the von Mises-Fisher (VMF) probability distribution, which serves as the prior distribution of the latent space of the variational autoencoder. Since the VMF distribution is defined on the unit hypersphere, such a non-parametric variational autoencoder based on the VMF distribution is called a non-parametric spherical variational autoencoder. The VMF mixture model constructed here is a synthesis of multiple weighted VMF probability distributions. In this method, each JPEG2025091403000054.jpg 44-dimensional electrocardiogram data vector JPEG2025091403000055.jpg 33 is generated by a certain random process, and each is assumed to be accompanied by a corresponding latent representation that cannot be directly observed. JPEG2025091403000056.jpg 33. This latent representation JPEG2025091403000057.jpg 33 can be regarded as the latent embedding (or code) obtained after the data vector JPEG2025091403000058.jpg 33 is processed by the encoder. Also, here, the latent representation Assume that JPEG2025091403000059.jpg33 is derived from an infinite VMF mixture model based on the Pitman-Yor process mixture model. Latent representation By assuming that JPEG2025091403000060.jpg33 follows an infinite VMF mixture model distribution, here the raw data Rather than directly performing cluster analysis on JPEG2025091403000061.jpg33, the latent representation of the ECG data in the latent space JPEG2025091403000062.jpg33 can be clustered. By adopting the framework of a nonparametric model based on the Pitman-Yor process mixture model, the number of categories in the latent space can be automatically adjusted as the amount of data increases. Specific embodiments include the following steps.

[0022] Step 1): Collecting the ECG dataset Let JPEG2025091403000063.jpg627 be the dataset containing the N collected ECG data, and each data in it JPEG2025091403000064.jpg44 is A 44-dimensional ECG data vector.

[0023] Step 2): Modeling the ECG data model using a nonparametric spherical variational autoencoder: First,[[]] Let JPEG2025091403000066.jpg527 be the observed data in the latent space As the representation of JPEG2025091403000067.jpg44, thus JPEG2025091403000068.jpg53 is Called the latent representation (or code) of JPEG2025091403000069.jpg44. Each latent representation Assume that both JPEG2025091403000070.jpg53 follow a non-parametric mixture model composed of an infinite number of VMF distributions. In the deep generative model constructed here, Samples from the 43rd cluster for JPEG2025091403000071.jpg43 To generate JPEG2025091403000072.jpg44 for the 44th cluster The selection of cluster JPEG2025091403000073.jpg43 is defined as follows, assuming that the parameter follows a Categorical Distribution with JPEG2025091403000074.jpg44 as the parameter, In Equation 627, JPEG2025091403000076.jpg717, JPEG2025091403000077.jpg46 is the mixing coefficient of cluster JPEG2025091403000078.jpg43 (i.e., the prior probability of cluster JPEG2025091403000079.jpg43), and satisfies the conditions of JPEG2025091403000080.jpg512 and JPEG2025091403000081.jpg719.

[0024] As shown in Figure 2, in this method, the prior probability of the mixing coefficient JPEG2025091403000082.jpg45 is constructed based on the Pitman-Yor process mixture model using the Stick-Breaking representation method. In the Pitman-Yor process mixture model based on the Stick-Breaking representation method, the mixing coefficient JPEG2025091403000083.jpg45 is shown as follows: In Equation 1131, JPEG2025091403000085.jpg620 follows a Beta distribution, and its expression form is as follows: In JPEG2025091403000086.jpg875, JPEG2025091403000087.jpg59 is a Beta distribution, JPEG2025091403000088.jpg54 is the discount parameter in the Pitman - Yor process mixture model, and JPEG2025091403000089.jpg517 satisfies the conditions, JPEG2025091403000090.jpg54 is the density parameter, and JPEG2025091403000091.jpg515 satisfies the conditions.

[0025] Here, assuming that the latent space follows a non - parametric VMF mixture model in advance, When JPEG2025091403000092.jpg43 is selected, The latent representation of JPEG2025091403000093.jpg44 JPEG2025091403000094.jpg53 is obtained through the following process by sampling from its latent distribution: In JPEG2025091403000095.jpg1381, JPEG2025091403000096.jpg611, JPEG2025091403000097.jpg612, JPEG2025091403000098.jpg617 is a VMF distribution, JPEG2025091403000099.jpg54 and JPEG2025091403000100.jpg44 are respectively the Position parameter and concentration parameter of the 43rd VMF probability distribution in the non - parametric VMF mixture model, and the function JPEG2025091403000102.jpg512 is the corrected JPEG2025091403000103.jpg49 is the first kind of Bessel function of order one. Here, the cluster JPEG2025091403000104.jpg43 samples from the latent representation JPEG2025091403000105.jpg53 is obtained, and a probability decoder that also follows the VMF probability distribution JPEG2025091403000106.jpg614, as shown by the following equation, generates a sample JPEG2025091403000107.jpg44, JPEG2025091403000108.jpg638 In the equation, the parameters JPEG2025091403000109.jpg56 and JPEG2025091403000110.jpg44 are such that when the input is JPEG2025091403000111.jpg53 and the parameters are JPEG2025091403000112.jpg45 of the neural network JPEG2025091403000113.jpg513 are automatically obtained as shown by the following equation by training the neural network JPEG2025091403000114.jpg635 In the equation, the neural network JPEG2025091403000115.jpg513 may be set to any network type according to different application scenarios. In the present invention, a long short-term memory network (LSTM) is adopted.

[0026] The above-mentioned sample JPEG2025091403000116.jpg44 is generated according to the generation procedure. Here, the deep generation model is defined as follows JPEG2025091403000117.jpg6128Similar to the ordinary variational autoencoder, based on the idea of variational inference, the objective function of the deep generative model here is JPEG2025091403000118.jpg513the variational lower bound (also called the evidence lower bound, ELBO), which is obtained by the following formula JPEG2025091403000119.jpg17128where JPEG2025091403000120.jpg58represents the expectation calculation JPEG2025091403000121.jpg620is the true posterior distribution JPEG2025091403000122.jpg520is the variational posterior distribution that approximates it. According to the Mean-Field Theory, the variational posterior distribution JPEG2025091403000123.jpg620can be further factorized into JPEG2025091403000124.jpg665where JPEG2025091403000125.jpg612is the probability encoder and follows the VMF distribution as follows JPEG2025091403000126.jpg636where the parameters JPEG2025091403000127.jpg64and JPEG2025091403000128.jpg44are obtained automatically by training the deep neural network JPEG2025091403000129.jpg44with the input being JPEG2025091403000130.jpg54and the parameters being JPEG2025091403000131.jpg512as shown by the following formula JPEG2025091403000132.jpg630the decoder JPEG2025091403000133.jpg512is similar to the encoder JPEG2025091403000134.jpg also selects and employs the LSTM neural network with 512.

[0027] From the above definition, the objective function of the proposed deep generative model based on the hypersphere latent space can be redefined as follows. JPEG2025091403000135.jpg 21128 where JPEG2025091403000136.jpg 652 is the KL divergence between the variational posterior distribution JPEG2025091403000137.jpg 620 and JPEG2025091403000138.jpg 616.

[0028] Here, by adopting the Stochastic Gradient Variational Bayes (SGVB) inference method and the ADAM optimizer, the objective function ELBO is updated and optimized.

[0029] As shown in Figure 4, JPEG2025091403000139.jpg 613 of the variational posterior distribution JPEG2025091403000140.jpg 612 to calculate the expected value JPEG2025091403000141.jpg 628, and assign the data JPEG2025091403000142.jpg 44 to the cluster with the maximum probability to complete the clustering. Comparative Example

[0030] In the most relevant prior art [1], a method for clustering data by combining a variational autoencoder and a Gaussian mixture model has been proposed. This method assumes that the latent representation of each data vector follows a Gaussian mixture model and transforms the clustering problem of raw data into a cluster analysis of the latent representation. This method trains the model by stochastic gradient variational Bayes. However, one of the limitations of this method is that the number of categories needs to be manually set in advance, which somewhat limits its flexibility and adaptability. In comparison, the method of the present invention can more effectively handle complex and large-scale electrocardiogram data analysis tasks. [1]Z. Jiang, Y. Zheng, H. Tan, B. Tang, and H. Zhou, "Variational deep embedding: An unsupervised and generative approach to clustering," in Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2017, pp. 1965-1972.

[0031] The present invention proposes a new solution under the technical conditions proposed by the prior art [1].

[0032] First, the clustering model in the prior art [1] assumes that the latent representation of each data vector follows a Gaussian mixture model. However, recent research has shown that in a clustering model based on a deep autoencoder, when the latent representation of the raw data is L2-normalized, the clustering performance is significantly improved. The L2-normalized data corresponds to points located on the unit hypersphere and is generally called spherical data. Also, in L2-normalization, regularization can be introduced into the modeling of the data to restrict the representation space to the unit hypersphere, thereby effectively preventing overfitting of the model. For spherical data, the von Mises-Fisher (VMF) distribution has a probability density defined on the unit hypersphere and is more suitable as a modeling tool for such data than the Gaussian distribution. Therefore, the present invention adopts a prior distribution based on the VMF mixture model to construct a spherical variational autoencoder, thereby optimizing the modeling of the latent representation.

[0033] Next, the Gaussian mixture model described in the prior art [1] is a finite mixture model that requires the number of mixture components (i.e., the number of clusters in clustering) to be preset. In contrast, the VMF mixture model in the present invention is a nonparametric Bayesian mixture model constructed based on the Pitman-Yor process. In a clustering problem, the nonparametric Bayesian mixture model assumes that the data is generated from a mixture model composed of an infinite number of probability distributions and automatically determines the number of mixture components (i.e., the number of clusters) during the training of the model. This method enhances the flexibility and adaptability of the model, enables the model to more accurately reflect the latent structure of the data, and thereby improves the clustering effect.

[0034] To evaluate the effectiveness of the method according to the present invention, it is compared here with two conventional clustering methods, K-means and Gaussian mixture model (GMM), and a clustering method based on a variational autoencoder using the Gaussian mixture model [1] (abbreviated as VaDE). For each method, 10 repeated experiments were conducted, and the average ACC value was used as an indicator for performance comparison. The experimental results are shown in Table 1. From the comparison results, it can be seen that the present invention is excellent in clustering ECG data, achieving a higher ACC value compared to the prior art and showing its superiority.

[0035] Table 1 JPEG2025091403000143.jpg3464

[0036] In summary, according to the present invention, in the clustering problem, the nonparametric Bayesian mixture model assumes that the data is generated from a mixture model composed of an infinite number of probability distributions and automatically determines the number of mixture components during the training of the model. This method enhances the flexibility and adaptability of the model, enables the model to more accurately reflect the latent structure of the data, thereby improving the clustering effect. A more efficient, accurate and flexible solution is provided for the cluster analysis of electrocardiogram data.

[0037] In addition, in this specification, the term "comprise", "include" or any other variant thereof is intended to include non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements specific to that process, method, article or device.

[0038] In this specification, the principles and embodiments of the present invention have been described using specific examples. However, the description of the above examples is only for the purpose of facilitating the understanding of the method of the present invention and its core idea. The above is only a preferred embodiment of the present invention. Due to the limitation of text expression, objectively, there are infinitely many specific structures. For those skilled in the art, on the premise of not departing from the principles of the present invention, some improvements, modifications, or changes can be made, and the above technical features can also be combined in an appropriate manner. It should be noted that these improvements, modifications, changes, combinations, or forms directly obtained by applying in other cases without improving the concept and technical solution of the present invention should be regarded as being within the protection scope of the present invention.

Claims

1. Electrocardiogram Data Vector 1) acquiring an ECG dataset including: Nonparametric Spherical Variational Autoencoder for VMF Distribution Using electrocardiogram data vector Processing the latent representation Step 2) to obtain Based on the non-parametric VMF mixture model, latent representation Probability of belonging to a category According to the ECG data vector Determine the category to which it belongs, teeth 3) representing the th cluster; and Probabilistic decoder according to VMF probability distribution 4) generating samples by 5) training the model by employing the stochastic gradient variational Bayesian inference method and the ADAM optimizer to update and optimize the objective function ELBO; Variational posterior distribution of Expectations for By calculating 6) assigning ,to ,the ,clustering ,to ,complete ,the ,clustering; A method for clustering ECG data based on a non-parametric spherical variational autoencoder, comprising:

2. In the step 1) of acquiring an ECG data set, Let be a data set including N collected ECG data, and each data The method for clustering ECG data based on non-parametric spherical variational autoencoder according to claim 1, characterized in that, x, is a D-dimensional ECG data vector.

3. The non-parametric spherical variational autoencoder is as follows: In the formula, the parameters and means that the input is And the parameters are Deep Neural Network The method for clustering ECG data based on a non-parametric spherical variational autoencoder according to claim 1, characterized in that it is obtained by training:

4. Step 3: Latent Representation can be obtained by sampling from the latent distribution through the process During the ceremony, 、 、 is the VMF distribution, and are the nonparametric VMF mixture models, are the location and concentration parameters of the VMF probability distribution, and are functions has been fixed The following Bessel function of the first kind: During the ceremony, 、 is a cluster is the mixing coefficient of and Fulfill the conditions of The mixing coefficient The prior probability of is constructed based on the Pitman-Yor process mixture model using the Stick-Breaking representation method, in which the mixture coefficients is shown as follows: ； During the ceremony, follows the Beta distribution and has the following representation: ； During the ceremony, is the Beta distribution, is the discount parameter in the Pitman-Yor process mixture model, and Fulfill the conditions of is the density parameter, and The method for clustering ECG data based on a non-parametric spherical variational autoencoder according to claim 1, characterized in that the following condition is satisfied:

5. A probability decoder according to the VMF probability distribution is as follows: ； In the formula, the parameters and means that the input is And the parameters are Neural Network The method for clustering ECG data based on a non-parametric spherical variational autoencoder according to claim 1, characterized in that it is obtained by training:

6. The method for clustering ECG data based on a non-parametric spherical variational autoencoder according to claim 1 , characterized in that the objective function is as follows:

Images