Arrhythmia Classification Method, Device, and Medium Based on Multimodal Diffusion Model

Abstract

This application relates to the interdisciplinary research technical field of biomedicine and computer science, and relates to an arrhythmia classification method, device, and medium based on a multimodal diffusion model. The method includes: preprocessing the input original electrocardiogram signal to generate three-modal data as training set data; constructing a personalized arrhythmia classification model; freezing the parameters of the diffusion-driven feature extractor, inputting the personalized optimized training set data into the personalized arrhythmia classification model, and training the module; collecting the electrocardiogram signal to be detected, preprocessing it into three modalities of time-domain waveform, short-time Fourier transform spectrogram, and Lorenz scatter plot, and performing personalized optimization; inputting the optimized data into the trained personalized arrhythmia classification model to obtain the classification result. The present invention realizes high-precision, low-labeling-dependent, and individual-adaptive arrhythmia judgment.

Description

Technical Field

[0001] The present application relates to the interdisciplinary research technology field of biomedicine and computer science, and relates to arrhythmia classification method, device, and medium based on a multimodal diffusion model. Background Art

[0002] Currently, arrhythmias refer to cardiovascular dysfunction caused by disturbances in myocardial electrophysiological activity, primarily manifesting as clinical symptoms such as irregular heart rhythm, irregular rate, or conduction block. The increasing aging population and changes in modern lifestyles have not only led to a continuous increase in the overall incidence of these conditions, but also a trend of their prevalence spreading to younger age groups. Medical classifications include two main types of arrhythmias based on the site of origin: supraventricular (involving the sinoatrial node, atria, and atrioventricular junction) and ventricular. Essentially, the rhythmic contraction and relaxation of the myocardium are precisely regulated by bioelectrical impulses. These electrical signals propagate in an orderly manner through specialized conduction pathways, driving the heart's cyclical pumping function. When this electrophysiological activity is disrupted, the myocardial contraction rhythm becomes distorted, resulting in various clinical manifestations of arrhythmias. Accurate diagnosis of arrhythmias is crucial for preventing cardiovascular diseases such as myocardial infarction, coronary heart disease, and sudden cardiac death.

[0003] Traditional judgment methods rely on expert experience to manually interpret the time-domain waveform, spectral characteristics, and morphological properties of electrocardiogram (ECG) signals, resulting in low efficiency and poor consistency. Automatic detection algorithms based on medical features have poor generalization, and due to individual differences in physiological characteristics, individual parameter adjustments are required for each patient, which relies heavily on professional knowledge and clinical experience. While existing deep learning-based classification methods have made some progress, supervised learning methods require a large amount of labeled data, and ECG data labeling is labor-intensive and inefficient.

[0004] To address the data labeling challenges of supervised learning, a self-supervised learning method based on contrastive learning constructs positive and negative sample pairs and uses unlabeled ECG data to learn discriminative feature representations. This reduces labeling dependency while improving the model's ability to extract waveform features. Existing techniques typically use data augmentation strategies such as random cropping, noise addition, or time-frequency transformation to generate contrast samples. This allows the model to capture potential abnormal heart rhythm patterns by maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs. However, such methods still have significant flaws: first, ECG signals have strict timing dependence and local sensitivity to pathological features. Random data augmentation operations can easily destroy the morphology and periodicity of key waveform segments (such as P waves and QRS complexes), causing the model to learn non-robust features; second, the feature differences between arrhythmia types are subtle (such as supraventricular premature contractions and ventricular premature beats), and the contrastive learning strategy of aligning positive samples in the latent space may over-smooth the category boundaries, making it difficult to effectively distinguish similar pathological signals; in addition, existing methods have the problem of inconsistent goals between pre-training and downstream fine-tuning. The contrastive loss function optimized in the pre-training stage fails to be directly related to the arrhythmia classification task, resulting in reduced feature transfer efficiency, especially in scenarios with data distribution offset or noise interference. The model performance is significantly degraded.

[0005] The emergence of generative models offers a new approach to addressing these issues. They generate synthetic samples by modeling data distributions, enhancing the model's adaptability to data noise and distribution shifts. The Denoising Diffusion Probabilistic Model (DDPM), a new generative network, learns the essential characteristics of data through a progressive denoising and denoising process, demonstrating excellent feature representation capabilities in fields such as images and speech. The core advantage of the diffusion model lies in its multi-scale noise prediction mechanism, which separates signal and noise components during the denoising process, thereby extracting more discriminative high-level semantic features.

[0006] In ECG signal analysis, multi-modal data fusion is an important approach to improving the performance of arrhythmia classification. Specifically, the time-domain waveform of ECG can accurately capture dynamic features such as the amplitude, duration, and ST-segment deviation of the electrocardiogram signal, intuitively reflecting the conduction abnormalities in each stage of cardiac electrical activity; frequency-domain features can effectively extract key information such as heart rate variability and the periodicity of the rhythm after high-frequency noise suppression through Fourier transform or wavelet analysis, and are particularly suitable for identifying rhythm disorders with specific spectral features such as atrial fibrillation; while the morphological scatter plot constructed based on the RR interval maps the time-series signal into a two-dimensional spatial distribution, and by quantifying the statistical characteristics of the differences between adjacent heartbeat intervals, it can intuitively reveal the abnormal cardiac autonomic regulation in pathological states such as sinus arrhythmia and premature ventricular contractions. However, existing methods mostly process single modalities independently and lack a collaborative optimization mechanism for cross-modal features, resulting in insufficient information utilization. The multi-scale feature extraction characteristics of the diffusion model provide a natural advantage for multi-modal fusion, which can adaptively integrate time-frequency-spatial features of different modalities through a hierarchical denoising process, enhancing the model's ability to represent complex cardiac rhythm patterns.

[0007] For the need of individualized judgment, traditional deep learning models usually adopt the full-model fine-tuning strategy, that is, based on a pre-trained model, a small amount of individual-specific ECG data is used to re-adjust the network weight parameters to adapt to the feature distribution shift caused by the physiological state, electrode contact impedance, or pathological development stage of different patients. Although such methods can improve the classification accuracy of the model on specific individuals, the computational cost is extremely high, and overfitting problems are likely to occur in scenarios with scarce data. To break through this technical bottleneck, the noise scheduling mechanism of the diffusion model provides a new idea for solving this problem: by dynamically adjusting the noise addition step size, the intensity of noise injection during the feature extraction process can be controlled, so as to adapt to the individual data distribution differences without modifying the network parameters, significantly reducing the computational complexity of personalized adaptation.

[0008] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0009] To have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. The summary is not a general review, nor is it intended to identify key / important elements or delineate the protection scope of these embodiments, but rather serves as a preface to the subsequent detailed description.

[0010] The embodiments of the present disclosure provide an arrhythmia classification method, device, and medium based on a multi-modal diffusion model, which realize high-precision, low-labeling-dependent, and individual-adaptive automatic judgment of arrhythmia through diffusion-driven feature extraction, cross-modal attention fusion, and noise step-size-driven personalized optimization strategies.

[0011] In some embodiments, the method includes:

[0012] Preprocess the input original electrocardiogram (ECG) signal to generate a short-time Fourier transform spectrogram and a Lorenz scatter plot, and use the three-modal data of the time-domain waveform of the original ECG signal as the training set data;

[0013] Construct a personalized arrhythmia classification model, including a diffusion-driven feature extractor, a cross-modal fusion module, and a classifier; input each modal data into the diffusion model feature extractor respectively to extract multi-scale robust features, dynamically integrate the features of the three modalities through the cross-modal attention fusion module, and output the arrhythmia probability through the classifier for the integrated features;

[0014] Freeze the parameters of the diffusion-driven feature extractor, input the personalized optimized training set data into the personalized arrhythmia classification model, train the parameters of the feature fusion module and the classifier part, and obtain the trained personalized arrhythmia classification model;

[0015] Collect the ECG signal to be detected, preprocess it into a time-domain waveform, a short-time Fourier transform spectrogram, and a Lorenz scatter Figure 3 plot of three modalities, and perform personalized optimization;

[0016] Input the optimized data into the trained personalized arrhythmia classification model to obtain the classification result.

[0017] Preferably, the diffusion-driven feature extractor adopts the downsampling encoding part and the middle bottleneck layer part in the pre-trained U-Net network;

[0018] Specifically, the downsampling encoding part includes a convolutional layer and three-level downsampling modules, and each level of downsampling module is composed of two residual network blocks in series; the middle bottleneck layer includes two residual network blocks and a multi-head self-attention module, and the self-attention mechanism of the multi-head self-attention module models the global dependence through query-key-value interaction.

[0019] Preferably, the pre-training method of the U-Net network is as follows:

[0020] Input the noisy training set into the U-Net network, and for each modality, perform pre-training through an unsupervised paradigm, using the KL divergence between the predicted noise and the true noise as the loss function,

[0021] The method of adding noise to the training set data is as follows:

[0022] ,

[0023] ,

[0024] Among them, represents the original input data, represents the noise randomly sampled from the standard normal distribution, represents at the time step a decay coefficient, represents the noise intensity at each time step.

[0025] Preferably, the specific implementation of the cross-modal attention fusion module is as follows:

[0026] After the time-domain waveform modality, short-time Fourier transform spectrogram modality, and Lorenz scatter plot modality data are independently encoded, the feature of each modality generates a query vector, a key vector, and a value vector respectively;

[0027] The weights of each modality are obtained through query-key interaction calculation and then normalization; the features are fused according to the weights of each modality.

[0028] Preferably, the personalized optimization method adopts Bayesian optimization to select the noise addition step size for each modality data;

[0029] The personalized optimization method includes:

[0030] Define the parameter set, parameter space, and optimization rounds of the noise addition step size for each modality;

[0031] Use Gaussian process to construct a surrogate function;

[0032] Randomly initialize the parameter set as the initial observation point;

[0033] Calculate the optimization index corresponding to the observation point, and update the surrogate function according to all known observation points;

[0034] Based on the current surrogate function, use the upper confidence bound strategy to construct an acquisition function, select the next observation point according to the acquisition function, and observe the optimization index for this observation point;

[0035] Iterate the above process until the preset optimization rounds are reached, and output the optimal parameter set.

[0036] Preferably, calculate the optimization index corresponding to the observation point, and update the surrogate function according to all known observation points. The specific method is as follows:

[0037] Perform two rounds of fine-tuning on the current parameter set for the personalized arrhythmia classification model, calculate the loss values of the two rounds of fine-tuning based on the weighted cross-entropy loss function, and generate the optimization index value in combination with the model weight parameters , and the specific formula is as follows:

[0038] ,

[0039] Among them, Denote the loss function value of the model at the th round, Denote the loss function value of the model at the th round, Denote the weight parameter of the model at the th round;

[0040] Update the surrogate function according to all known observation points;

[0041] After the update is completed, reset the personalized arrhythmia classification model to the initial state.

[0042] Preferably, the training process of the personalized arrhythmia classification model is as follows:

[0043] Freeze the parameters of each modal diffusion-driven feature extractor;

[0044] Use the personalized optimization method to add noise to the data of each modality, and then send it to the modal diffusion-driven feature extractor for feature extraction;

[0045] Input the extracted features into the modal attention fusion module for fusion, and input the fused features into the classifier. Train the personalized arrhythmia classification model using weighted cross-entropy loss. The weighted cross-entropy loss is as follows:

[0046] ,

[0047] where, is the inverse frequency weight of class , represents the one-hot encoding of the true label of class , represents the probability that the model predicts this data belongs to class , represents the total number of arrhythmia classes.

[0048] Preferably, the electrocardiogram signal to be detected is first segmented, with each segment having a length of 10 seconds, an overlap rate of 50%, and a sampling rate of 250 Hz.

[0049] In some embodiments, the device includes a processor and a memory storing program instructions. The processor is configured to execute the arrhythmia classification method based on the multi-modal diffusion model when running the program instructions.

[0050] A computer-readable storage medium stores a computer program, which when executed by a processor implements the arrhythmia classification method based on the multi-modal diffusion model.

[0051] The arrhythmia classification method, device, and medium based on a multi-modal diffusion model provided by the embodiments of the present disclosure can achieve the following technical effects:

[0052] (1) A feature extraction architecture based on a multi-modal diffusion model is proposed. Using the unsupervised training paradigm of the diffusion model, the noise components and multi-scale temporal features in the ECG signal are separated through a hierarchical denoising process, constructing a robust heart rhythm representation while reducing the dependence on data annotation.

[0053] (2) A cross-modal attention fusion module is designed to jointly fuse multi-source features of the time-domain waveform, spectrogram, and Lorenz scatter plot, realizing dynamic weight allocation and collaborative optimization of the feature space.

[0054] (3) A personalized optimization strategy driven by the noise step size is constructed. By using a zero-order optimization method, the noise addition intensity of the input data is dynamically adjusted to adapt to the differential expression of individual ECG signals.

[0055] (4) The neural network architecture search technology is used to optimize the feature extraction network structure, removing redundant information paths and retaining multi-granularity feature abstraction layers, improving the model calculation efficiency and generalization ability.

[0056] The above general description and the following description are only exemplary and explanatory, and are not used to limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] One or more embodiments are exemplarily illustrated by the corresponding drawings. These exemplary illustrations and the drawings do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a scale limitation, and among them:

[0058] Figure 1 is a schematic diagram of the method flow of the present invention;

[0059] Figure 2 is a schematic diagram of the overall architecture of the method of the present invention;

[0060] Figure 3 is a schematic diagram of the U-Net structure provided by the present invention;

[0061] Figure 4 is a schematic diagram of the conversion of the ECG signal - Lorenz scatter plot of the present invention;

[0062] Figure 5 is a schematic diagram of the surrogate function and acquisition function of the Bayesian optimization process of the present invention;

[0063] Figure 6 is a schematic diagram of the device structure provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] To more comprehensively understand the features and technical content of the embodiments of the present disclosure, the implementation of the embodiments of the present disclosure will be elaborated in detail below with reference to the accompanying drawings. The attached drawings are for reference and illustration only and are not intended to limit the embodiments of the present disclosure. In the following technical descriptions, for the sake of explanation, numerous details are provided to give a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices may be shown in a simplified manner to simplify the drawings.

[0065] In the description of the embodiments of the present disclosure, the terms "first", "second", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so as to implement the embodiments of the present disclosure described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0066] Unless otherwise specified, the term "plurality" means two or more.

[0067] In the embodiments of the present disclosure, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B.

[0068] The term "and / or" is a description of the association relationship of objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B these three relationships.

[0069] The term "corresponding" can refer to an association relationship or a binding relationship. A corresponding to B means that there is an association relationship or a binding relationship between A and B.

[0070] Embodiment 1

[0071] As Figure 1 shown, an arrhythmia classification method based on a multi-modal diffusion model realizes automatic judgment of arrhythmia with high robustness and low annotation cost by constructing a hierarchical feature extraction network, a multi-modal dynamic fusion mechanism, and a personalized optimization strategy driven by noise step size.

[0072] Including:

[0073] S1: Preprocess the input original electrocardiogram (ECG) signal to generate a short-time Fourier transform spectrogram, a Lorenz scatter plot, and the time-domain waveform of the original ECG signal as three-modal data for the training set.

[0074] S2: Construct a personalized arrhythmia classification model, including a diffusion-driven feature extractor, a cross-modal fusion module, and a classifier; input each modality data into the diffusion model feature extractor for hierarchical denoising processing to extract multi-scale robust features, dynamically integrate the features of the three modalities through the cross-modal attention fusion module, and output the arrhythmia probability through the classifier for the integrated features.

[0075] S3: Freeze the parameters of the diffusion-driven feature extractor, input the personalized optimized training set data into the personalized arrhythmia classification model, train the parameters of the feature fusion module and the classifier part, and obtain the trained personalized arrhythmia classification model.

[0076] S4: Collect the electrocardiogram signals to be detected, and preprocess them into time-domain waveforms, short-time Fourier transform spectrograms, and Lorenz scatter plots Figure 3 for the three modalities, and perform personalized optimization.

[0077] S5: Input the optimized data into the trained personalized arrhythmia classification model to obtain the classification result.

[0078] As a refinement of the above embodiment, as Figure 2 shown, the method in this embodiment includes three stages: multi-modal data preprocessing, personalized arrhythmia classification model construction, and model prediction.

[0079] The multi-modal data preprocessing is involved in both the training and prediction of the model. The original ECG signals all need to go through the multi-modal data conversion module to generate short-time Fourier transform spectrogram and Lorenz scatter plot modalities; specifically including:

[0080] (1) Short-time Fourier transform spectrogram modality:

[0081] This method uses the short-time Fourier transform to convert the original ECG signal into the frequency domain modality, and its process can be described by the following formula:

[0082] ,

[0083] where is a two-dimensional function defined in time and frequency, is the input original ECG segment, is the complex exponential function, representing the basis function of the Fourier transform, used to convert the time-domain signal to the frequency domain, is the angular frequency, is the center position of the current window, is the imaginary unit, represents the short-time Fourier transform operation, is the window function used to reduce the spectral energy leakage. The Hanning window is a commonly used window function, and its definition formula is as follows:

[0084] ,

[0085] Among them, is the number of sampling points, is the window length.

[0086] (2) Lorenz scatter plot modality:

[0087] The Lorenz scatter plot is a commonly used two-dimensional scatter plot in the field of cardiology, which shows the relationship between consecutive R-R intervals in an electrocardiogram. The coordinates of each point in the Lorenz scatter plot are calculated using the following formula:

[0088] ,

[0089] Among them, is defined as a series of RR intervals by locating all R peaks in the ECG segment. Refer to Figure 5 , is 0.8 - 0.5 = 0.3, is 0.5 - 1.0 = -0.5, then the coordinates of this point are (0.3, -0.5).

[0090] The personalized arrhythmia classification model consists of a diffusion-driven feature extractor, a cross-modal fusion module, and a personalized classifier. After being preprocessed by the multi-modal data preprocessing module, the input ECG signal synchronously generates time-domain waveforms, short-time Fourier transform (STFT) spectrograms, and Lorenz scatter Figure 3 type modal data. Each modal data is respectively input into each modal diffusion feature extractor for noise injection and denoising processing, and robust representations are obtained through hierarchical feature extraction; the cross-modal fusion module dynamically integrates multi-source features using the attention mechanism, and finally the lightweight classifier outputs the arrhythmia category probability.

[0091] In the personalized arrhythmia classification model, the denoising network of each modality diffusion model uses a U-Net structure, which includes a symmetric downsampling encoding path, an intermediate bottleneck layer, and an upsampling decoding path. The feature maps of the same size between the encoding path and the decoding path are connected by a shortcut structure. Specifically, the input modality enters the encoding path through an initial one-dimensional convolutional layer (the number of channels is expanded to 48), and the size of the feature map is gradually compressed and the channel dimension is increased through three levels of downsampling modules: the first level keeps the number of channels at 48, the second level is increased to 96, and the third level maintains 96 channels. Each level of the downsampling module is composed of two residual network blocks in series. Each residual block integrates weight-normalized convolution (convolution kernel size 3, stride 1), group normalization (number of groups 8), and the SiLU activation function, and dynamically adjusts feature scaling and offset through a time embedding vector; at the end of each level, the size of the feature map is halved through a convolution with a stride of 2 (except for the last level). The intermediate bottleneck layer contains two residual network blocks and a multi-head self-attention module, and the number of channels remains 96. Among them, the self-attention mechanism models global dependencies through query-key interactions, enhancing the ability to capture long-range patterns of heart rhythms. The decoding path uses three levels of upsampling modules. At each level, the size of the feature map is restored through transposed convolution and the number of channels is gradually reduced (96→48). At the same time, the feature maps of the corresponding levels of the encoding path are concatenated in channels, and multi-scale information is fused through residual blocks and linear attention modules. The final output layer aggregates shallow and deep layer features through residual blocks, and maps the number of channels to the target dimension through 1×1 convolution.

[0092] Since the U-Net network trained for the generation task introduces a mismatch between high-frequency noise and semantic abstraction, it may lead to a performance decline in the classification task. For the diffusion-driven feature extractor in the proposed method, the downsampling encoding part and the intermediate bottleneck layer part of the pre-trained denoising U-Net network are used. These modules capture the most abstract and high-level features from the input data while consolidating rich hierarchical features.

[0093] The specific implementation of the cross-modal attention fusion module is as follows:

[0094] In view of the heterogeneous characteristics of multi-modal data, a cross-modal attention fusion module is designed. After the time-domain, frequency-domain, and morphological features are independently encoded, query vectors , key vectors , and value vectors are generated respectively. The cross-modal weights are calculated through query-key interactions:

[0095] For modality , its weight :

[0096] ,

[0097] where Represents the feature dimensions of the query vector and the key vector.

[0098] After normalization, the contribution degrees of each modality are obtained.

[0099] Final fused feature , which is mapped to a 128-dimensional high-level representation through a fully connected layer. This mechanism enables the model to adaptively focus on key modalities. For example, in atrial fibrillation detection, it strengthens the periodic anomaly features of spectrograms, and in ventricular premature beat classification, it emphasizes the QRS distortion of time-domain waveforms.

[0100] As a refinement of the above embodiment, in the pre-training stage of the U-Net network, a large-scale unlabeled ECG data is used. First, through a data processing module, the original unlabeled ECG dataset is converted to generate a corresponding short-time Fourier transform spectrogram and Lorenz scatter plot modality dataset. Then, for each modality, through an unsupervised paradigm, the denoising diffusion probability model is trained with minimizing the loss function as the optimization objective. The minimizing loss function is as follows:

[0101] ,

[0102] Where, is the noise randomly sampled from the standard normal distribution, represents the noisy data at time step t, represents the expectation of the following variables: time step t, real data sample , noise epsilon; are the model parameters, is the predicted output of the model.

[0103] Specifically, for each input training data, different degrees of noise addition are randomly performed, and the degree of noise addition is controlled by the noise addition step , and the noise addition function is defined as follows:

[0104] ,

[0105] ,

[0106] Where, represents the original input data, represents the noise randomly sampled from the standard normal distribution, represents a decay coefficient at time step t, represents the noise intensity at each time step.

[0107] After that, the denoising network in the diffusion model predicts the added noise, calculates the KL divergence (Kullback-Leibler Divergence) between the predicted noise and the real noise, and uses it as the loss function to optimize the denoising network. After the modal denoising diffusion probability model is trained, a partial structure of the denoising network is extracted as the diffusion-driven feature extractor.

[0108] As a refinement of the above embodiment, to adapt to individual ECG differences, a personalized optimization method driven by noise step size based on Bayesian optimization is proposed to select appropriate data noise addition step sizes for each modality. Define a personalized parameter set , which controls the noise injection intensity of each modality respectively; represents the time-domain waveform modality of the original ECG signal with added noise, represents the short-time Fourier transform spectrogram modality with added noise, represents the Lorenz scatter plot modality with added noise. In order to evaluate the model convergence ability with as few training rounds as possible, an original optimization metric is proposed to characterize the loss decrease efficiency of the model in the initial training stage, which is defined as follows:

[0109] ,

[0110] Among them, represents the value of the loss function of the model in the -th round, represents the value of the loss function of the model in the -th round, represents the weight parameter of the model in the -th round;

[0111] Select to use Gaussian Process to construct a surrogate function to approximate the objective function. Gaussian process is a class of stochastic processes , and its any n-dimensional distribution follows a multivariate normal distribution, that is: for any finite number of , any linear combination of is a normal distribution. Just as a normal distribution can be determined by specifying the mean and variance, a Gaussian process can be uniquely determined by specifying the mean function and the covariance function . In this method, the objective function is the mapping from the parameter set t to the proposed original optimization objective , denoted as , and the acquisition function constructed by the Gaussian process is defined as follows:

[0112] ,

[0113] Among them, is the mean value of the Gaussian function, which is set to a constant 0, is the covariance function. In this method, the radial basis function is selected as the covariance function, and its definition is as follows:

[0114]

[0115] Among them, is the length scale, and the larger its value, the stronger the exploration ability, indicates that the covariance function is used as the radial basis function.

[0116] The process of constructing the surrogate function using the Gaussian process is actually using the Maximum A Posteriori estimate (MAP). When the objective function of the known observation points is known, the function values of other parameter combinations in the parameter space in the objective are estimated, and the process is defined as follows:

[0117]

[0118]

[0119] Among them, represents the parameter combination, ) represents the probability distribution of the parameter combination under the actual observed value of the obtained objective function. Using the Bayesian formula for conversion, it is:

[0120] ,

[0121] Among them, is the probability of obtaining the observed value given the parameters, is the prior probability of the parameter values. In this method, assuming that all parameter combinations have an equal probability distribution, the above MPA degenerates into the Maximum Likelihood Estimate (MLE). The surrogate function constructed using the Gaussian function essentially reflects the distribution probability of the value corresponding to each parameter combination (which can be referred to Figure 5 ), and the distribution probability of the value corresponding to each parameter combination at the function point follows the Gaussian distribution and only converges at the observed points (because there are actual objective function values after observation).

[0122] In the Bayesian optimization process, except for the initial observation points, the subsequent observation points are determined by the acquisition function. The acquisition function is generally established based on the current surrogate function. This method uses the upper confidence bound strategy to construct the acquisition function (as shown in Figure 5 ), that is, select the maximum value corresponding to each parameter combination estimated by the surrogate function to construct the function , which is defined as follows:

[0123] ,

[0124] Among them, is the covariance matrix of the Gaussian process, defined as .

[0125] Specifically, the personalized optimization method includes:

[0126] (1) Define the set of noise-adding step parameters, parameter space, and the number of optimization rounds for each modality;

[0127] (2) Use the Gaussian process to construct a surrogate function;

[0128] (3) Randomly initialize the parameter set as the initial observation point;

[0129] (4) Calculate the optimization metrics corresponding to the observation points, and update the surrogate function based on all known observation points;

[0130] (5) Based on the current surrogate function, construct an acquisition function using the upper confidence bound strategy, select the next observation point according to the acquisition function, and observe the optimization metrics for this observation point;

[0131] (6) Iterate the process of (4)-(5) until the preset number of optimization rounds is reached, and output the optimal parameter set.

[0132] The above process is the same as the training process of the personalized arrhythmia classification model, but is only used to evaluate the training effect of the model under the current parameter set, and only two rounds of fine-tuning are performed, and the loss function values and model weight parameters obtained during the fine-tuning process are used to calculate the optimization objective , and the loss function uses the weighted cross-entropy function, which is the same as in the fine-tuning stage. After the evaluation of the current observation point, the updated model is not saved, and each observation during the optimization process starts from the initial state of the model.

[0133] This process only requires 2 iterations to effectively evaluate the performance of the model under the current parameter settings, reducing the computational cost by more than 90% compared to the traditional objective.

[0134] As a refinement of the above embodiment, during the training process of the personalized arrhythmia classification model, the parameters of the diffusion-driven feature extractor for each modality are frozen, and only the parameters of the feature fusion module and the classifier part are trained.

[0135] Specifically as follows:

[0136] Freeze the parameters of the diffusion-driven feature extractor for each modality;

[0137] Add noise to each modality data using the personalized optimization method, and then send it to the modality diffusion-driven feature extractor for feature extraction;

[0138] Input the extracted features into the modality attention fusion module for fusion, and input the fused features into a classifier (with a hidden layer dimension of 64 and ReLU activation). Train the personalized arrhythmia classification model using weighted cross-entropy loss. The weighted cross-entropy loss is as follows:

[0139] ,

[0140] where, is the inverse frequency weight of class , represents the one-hot encoding of the true label of class , represents the probability that the model predicts this data belongs to class , represents the total number of arrhythmia classes. Use weighted cross-entropy loss to overcome the imbalance of samples.

[0141] During the usage stage, the electrocardiogram signal to be detected is first segmented. Each segment has a length of 10 seconds, an overlap rate of 50%, and a sampling rate of 250 Hz. After multi-modal conversion, diffusion feature extraction, and fusion of each segment, the probability that the ECG segment belongs to each arrhythmia type is output.

[0142] It should be noted that: eliminate the interference of false negative samples through the multi-scale noise prediction mechanism of the diffusion model, reduce label dependence, and improve feature robustness; use cross-modal attention to achieve dynamic coordination of time-frequency-morphology features and enhance the representation ability of complex heart rhythms; explore the relationship between the noise addition step t and the model's personalization ability, and propose a noise step optimization strategy based on convergence sensitivity, replacing full model fine-tuning with zero-order search, and greatly reducing the personalization adaptation cost. Experiments show that this method improves the F1 value by 37.9% in cross-dataset testing and increases the personalization classification efficiency by 23.9%, providing reliable technical support for the accurate judgment of arrhythmias.

[0143] Next, in combination with the algorithm pseudocode, the detailed implementation process of the U-Net network pre-training, personalized optimization method, and personalized arrhythmia classification model training of the present invention will be further described in detail.

[0144] U-Net Network Pre-training Stage

[0145] In the pre-training stage of the present invention, the unsupervised training paradigm of the general denoising diffusion probability model is followed. The diffusion models are trained for the three modality data respectively, and the parameters of the denoising network part are extracted after the training is completed.

[0146] Input: Initial diffusion model denoising network weights , unlabeled general data , maximum noise addition step T, number of pre-training epochs EpochD.

[0147] Output: Feature extractor

[0148] for each modality

[0149]

[0150] #T is set to 1000

[0151]

[0152] for e in EpochsD:

[0153]

[0154] end

[0155] return

[0156] end

[0157] Personalized optimization method stage

[0158] The step-driven personalized optimization strategy proposed in the present invention is based on the Bayesian optimization algorithm, uses an original optimization objective to evaluate model performance, and the labeled data used in the optimization process does not exceed 1000 pieces. During this process, the personalized classification model proposed by the method does not update its parameters.

[0159] Input: Parameter set , maximum number of iterations N, surrogate model set , acquisition function set , diffusion-driven feature extractor , classifier weight parameters

[0160] Output: Optimal parameter set

[0161] 1: Randomly initialize the parameter set

[0162] 2: for step in range(2)

[0163] 3:

[0164] 4:

[0165] 5:

[0166] 6:

[0167] 7: Repeat 3 - 4

[0168] 8:

[0169] 9: return

[0170] 10: while

[0171] 11: Construct a surrogate model based on the currently obtained point set Build a surrogate model

[0172] 12: Based on the surrogate model , maximize the acquisition function to obtain the next evaluation point:

[0173] 13:

[0174] 14: Repeat 2 - 9

[0175] 15: end

[0176] 16: return

[0177] The noise - adding function of the above - mentioned algorithm is designed as follows, where is a decay coefficient at time step and is calculated through to obtain

[0178] Personalized arrhythmia classification model training phase

[0179] For each patient, the present invention uses no more than 300 labeled data for model fine - tuning to further improve the detection accuracy of the model on individual data. During the fine - tuning process, only the parameters of the classifier part are updated, and the parameters of the diffusion - driven feature extractor part are frozen, reducing the training cost.

[0180] Input: Optimal parameter set , maximum number of fine - tuning epochs EpochFT, diffusion - driven feature extractor , classifier weight parameters .

[0181] Output: Fine - tuned classifier weight parameters

[0182] for step in EpochFT # Set EpochFT to 20

[0183]

[0184]

[0185]

[0186] end

[0187] return

[0188] Combined Figure 6 As shown, an arrhythmia classification device 300 based on a multimodal diffusion model is provided in an embodiment of the present disclosure, including a processor 304 and a memory 301. Optionally, the device may further include a communication interface 302 and a bus 303. Among them, the processor 304, the communication interface 302, and the memory 301 can complete communication with each other through the bus 303. The communication interface 302 can be used for information transmission. The processor 304 can call the logical instructions in the memory 301 to execute the arrhythmia classification method based on the multimodal diffusion model in the above embodiment.

[0189] In addition, when the logical instructions in the above-mentioned memory 301 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0190] The memory 301, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 304 executes functional applications and data processing by running the program instructions / modules stored in the memory 301, that is, implements the arrhythmia classification method based on the multimodal diffusion model in the above embodiment.

[0191] The memory 301 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and applications required for at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 301 may include a high-speed random access memory and may also include a non-volatile memory.

[0192] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, and the computer-executable instructions are set to execute the above-mentioned arrhythmia classification method based on the multimodal diffusion model.

[0193] The above computer-readable storage medium may be a transient computer-readable storage medium or a non-transient computer-readable storage medium.

[0194] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements on some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An arrhythmia classification method based on a multimodal diffusion model, characterized in that It includes the following steps: Preprocess the input original electrocardiogram (ECG) signal to generate spectrograms of short-time Fourier transform and Lorenz scatter plots, along with the time-domain waveform of the original ECG signal. These three modal data are used as the training set data. Construct a personalized arrhythmia classification model, including a diffusion-driven feature extractor, a cross-modal fusion module, and a classifier. Input each modal data into the diffusion model feature extractor to extract multi-scale robust features. Dynamically integrate the features of the three modalities through the cross-modal attention fusion module, and output the arrhythmia probability through the classifier for the integrated features. Freeze the parameters of the diffusion-driven feature extractor. Input the personalized optimized training set data into the personalized arrhythmia classification model, and train the parameters of the feature fusion module and the classifier part to obtain the trained personalized arrhythmia classification model. Collect the ECG signal to be detected, preprocess it into three modalities: time-domain waveform, spectrogram of short-time Fourier transform, and Lorenz scatter plot, and perform personalized optimization. Input the optimized data into the trained personalized arrhythmia classification model to obtain the classification result. The personalized optimization method uses Bayesian optimization to select the noise addition step size for each modal data. The personalized optimization method includes: Define the parameter set, parameter space, and optimization rounds for the noise addition step size of each modality. Construct a surrogate function using Gaussian processes. Randomly initialize the parameter set as the initial observation point. Calculate the optimization metric corresponding to the observation point, and update the surrogate function based on all known observation points. Based on the current surrogate function, construct an acquisition function using the upper confidence bound strategy. Select the next observation point according to the acquisition function, and observe the optimization metric for this observation point. Iterate the above process until the preset optimization rounds are reached, and output the optimal parameter set.

2. The arrhythmia classification method based on a multi-modal diffusion model according to claim 1, wherein, The diffusion-driven feature extractor uses the downsampling encoding part and the middle bottleneck layer part in the pre-trained U-Net network. Specifically, the downsampling encoding part includes a convolutional layer and three levels of downsampling modules. Each level of downsampling module consists of two residual network blocks in series. The middle bottleneck layer includes two residual network blocks and a multi-head self-attention module. The self-attention mechanism of the multi-head self-attention module models the global dependence through query-key-value interaction.

3. The arrhythmia classification method based on a multimodal diffusion model according to claim 2, wherein The pre-training method of the U-Net network is as follows: Input the noisy training set into the U-Net network. For each modality, perform pre-training through an unsupervised paradigm, using the KL divergence between the predicted noise and the true noise as the loss function. The method of adding noise to the training set data is as follows: ， ， Among them, represents the original input data, represents the noise randomly sampled from the standard normal distribution, represents at the time step a decay coefficient, represents the noise intensity at each time step.

4. The arrhythmia classification method based on a multimodal diffusion model according to claim 1, wherein The specific implementation of the cross-modal attention fusion module is as follows: After the time-domain waveform modality, spectrogram of short-time Fourier transform modality, and Lorenz scatter plot modality data are independently encoded, the features of each modality generate query vectors, key vectors, and value vectors respectively. Calculate the weights of each modality through query-key interaction and then through normalization. Fusion the features according to the weights of each modality.

5. The arrhythmia classification method based on a multimodal diffusion model according to claim 1, wherein, Calculate the optimization metric corresponding to the observation point, and update the surrogate function based on all known observation points. The specific method is as follows: Perform two rounds of fine-tuning on the current parameter set for the personalized arrhythmia classification model, calculate the loss values of the two rounds of fine-tuning based on the weighted cross-entropy loss function, and generate an optimization metric value in combination with the model weight parameters , and the specific formula is as follows: ， Among them, represents the loss function value of the model in the th round, represents the loss function value of the model in the th round, represents the weight parameter of the model in the th round; Update the surrogate function based on all known observation points. After the update, reset the personalized arrhythmia classification model to the initial state.

6. The arrhythmia classification method based on a multi-modal diffusion model according to claim 5, wherein The training process of the personalized arrhythmia classification model is as follows: Freeze the parameters of the feature extractor driven by each modality diffusion; Use the personalized optimization method to add noise to the data of each modality, and then send it to the feature extractor driven by modality diffusion for feature extraction; The extracted features are input into the modal attention fusion module for fusion, and the fused features are input into the classifier. A personalized arrhythmia classification model is trained using weighted cross-entropy loss, and the weighted cross-entropy loss is as follows: ， Among them, is the inverse frequency weight of the class , represents the one-hot encoding of the true label of the class , represents the probability that this data predicted by the model belongs to the class , represents the total number of arrhythmia classes.

7. The arrhythmia classification method based on a multimodal diffusion model according to claim 1, characterized in that The electrocardiogram signal to be detected is first segmented, with each segment having a length of 10 seconds, an overlap rate of 50%, and a sampling rate of 250 Hz.

8. An arrhythmia classification device based on a multi-modal diffusion model, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the arrhythmia classification method based on the multi-modal diffusion model as described in any one of claims 1-7 when running the program instructions.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the program is executed by the processor, it implements the arrhythmia classification method based on the multi-modal diffusion model as described in any one of claims 1-7 above.

Images