Arrhythmia classification method and device based on multi-mode diffusion model, and medium

Abstract

The invention relates to the technical field of interdisciplinary research of biomedicine and computer science, and relates to an arrhythmia classification method and device based on a multi-modal diffusion model, and a medium. The method comprises the steps that input original electrocardiosignals are preprocessed, and three kinds of modal data are generated to serve as training set data; a personalized arrhythmia classification model is constructed; freezing diffusion driving feature extractor parameters, performing personalized optimization on training set data, inputting the data into a personalized arrhythmia classification model, and training the module; the method comprises the following steps: collecting an electrocardiosignal to be detected, preprocessing the electrocardiosignal into three modals of a time domain waveform, a short-time Fourier transform spectrogram and a Lorentz scatter diagram, and carrying out personalized optimization; and inputting the optimized data into the trained personalized arrhythmia classification model to obtain a classification result. According to the invention, high-precision, low-label-dependence and individual-adaptive arrhythmia judgment is realized.

Description

Technical Field

[0001] This application relates to the interdisciplinary research technical field of biomedicine and computer science, and relates to an arrhythmia classification method and device, and a medium based on a multimodal diffusion model. Background Art

[0002] At present, arrhythmia refers to the phenomenon of cardiovascular system dysfunction caused by myocardial electrophysiological activity disorders, mainly manifested as clinical symptoms such as abnormal heart rhythm, rate disorder, or conduction block. With the increasing aging of society and the transformation of modern lifestyle, the overall incidence of such diseases not only continues to rise, but the affected population is also showing a trend of gradually spreading to the younger population. According to the origin of the lesion, medical classification can be divided into two major types: supraventricular (covering the sinoatrial node, atrium, and atrioventricular junction area) and ventricular arrhythmia. In essence, the regular contraction and relaxation of the myocardium are precisely regulated by bioelectric pulses, and these electrical signals spread orderly through special conduction pathways, driving the heart to complete the periodic pumping function. When this electrophysiological activity is disordered, the myocardial contraction rhythm is dysregulated, thus forming various clinical manifestations of arrhythmia. The accurate judgment of arrhythmia is crucial for the prevention of cardiovascular diseases such as myocardial infarction, coronary heart disease, and sudden cardiac death.

[0003] Traditional judgment methods rely on expert experience to manually interpret the time-domain waveform, spectral characteristics, and morphological characteristics of electrocardiogram (ECG) signals, which have problems of low efficiency and poor consistency. The automatic detection algorithms based on medical characteristics have poor generalization ability, and due to the physiological characteristic differences among individuals, separate parameter adjustment is required for each patient, and this adjustment highly depends on professional knowledge and clinical experience. Although the existing deep learning-based classification methods have made some progress, the supervised learning methods require a large amount of labeled data support, while the manual cost of labeling ECG data is extremely high and the efficiency is extremely low.

[0004] To address the problem of data annotation in supervised learning, self-supervised learning methods based on contrastive learning construct positive and negative sample pairs and utilize unlabeled ECG data to learn discriminative feature representations, improving the model's ability to extract waveform features while reducing the dependence on annotation. Existing techniques typically use data augmentation strategies such as random cropping, adding noise, or time-frequency transformation to generate contrast samples, enabling the model to capture potential arrhythmia patterns by maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs. However, such methods still have significant drawbacks: First, the ECG signal has strict temporal dependence and local sensitivity to pathological features, and random data augmentation operations are likely to disrupt the morphology and periodic patterns of key waveform segments (such as P waves and QRS complexes), leading the model to learn non-robust features. Second, the feature differences between arrhythmia types are subtle (such as supraventricular premature contractions and ventricular premature beats), and the strategy of aligning positive samples in the latent space by contrastive learning may over-smooth the class boundaries, making it difficult to effectively distinguish similar pathological signals. In addition, there is a problem of inconsistent objectives in the pre-training and downstream fine-tuning phases of existing methods. The contrastive loss function optimized in the pre-training phase is not directly related to the arrhythmia classification task, resulting in reduced feature transfer efficiency and a significant decline in model performance, especially in scenarios of data distribution shift or noise interference.

[0005] The emergence of generative models provides a new idea for solving the above problems. They generate synthetic samples by modeling the data distribution, enhancing the model's adaptability to data noise and distribution shift. Among them, the Denoising Diffusion Probabilistic Model (DDPM), as an emerging generative network, learns the essential features of data through a progressive noise addition and denoising process, demonstrating excellent feature representation capabilities in fields such as images and speech. The core advantage of diffusion models lies in their multi-scale noise prediction mechanism, which can separate signal and noise components during the denoising process, thereby extracting more discriminative high-level semantic features.

[0006] In ECG signal analysis, multimodal data fusion is an important approach to enhancing the performance of arrhythmia classification. Specifically, the time-domain waveform of the ECG can accurately capture dynamic features such as the amplitude, duration, and ST-segment deviation of the electrocardiogram signal, intuitively reflecting the conduction abnormalities in each stage of cardiac electrical activity; frequency-domain features can effectively extract key information such as heart rate variability and the rhythm periodicity after high-frequency noise suppression through Fourier transform or wavelet analysis, and are particularly suitable for identifying rhythm disorders with specific spectral features such as atrial fibrillation; while the morphological scatter plot constructed based on the RR interval maps the time-series signal into a two-dimensional spatial distribution, and by quantifying the statistical characteristics of the differences between adjacent heartbeat intervals, it can intuitively reveal the abnormal cardiac autonomic regulation in pathological states such as sinus arrhythmia and premature ventricular contractions. However, existing methods mostly independently process a single modality and lack a collaborative optimization mechanism for cross-modal features, resulting in insufficient information utilization. The multi-scale feature extraction property of the diffusion model provides a natural advantage for multimodal fusion, which can adaptively integrate time-frequency-space features of different modalities through a hierarchical denoising process, enhancing the model's ability to represent complex cardiac rhythm patterns.

[0007] For the need of individualized judgment, traditional deep learning models usually adopt the full-model fine-tuning strategy, that is, based on a pre-trained model, a small amount of individual-specific ECG data is used to readjust the network weight parameters to adapt to the feature distribution shift caused by the physiological state, electrode contact impedance, or pathological development stage of different patients. Although such methods can improve the classification accuracy of the model on specific individuals, the computational cost is extremely high, and overfitting problems are likely to occur in scenarios with scarce data. To break through this technical bottleneck, the noise scheduling mechanism of the diffusion model provides a new idea for solving this problem: by dynamically adjusting the noise addition step size, the intensity of noise injection during the feature extraction process can be controlled, so as to adapt to the individual data distribution differences without modifying the network parameters, significantly reducing the computational complexity of personalized adaptation.

[0008] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0009] To have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. The summary is not a general review, nor is it intended to identify key / important constituent elements or delineate the protection scope of these embodiments, but rather serves as a preamble to the subsequent detailed description.

[0010] The embodiments of the present disclosure provide an arrhythmia classification method, device, and medium based on a multimodal diffusion model, which achieve high-precision, low-labeling-dependent, and individual-adaptive automated judgment of arrhythmias through diffusion-driven feature extraction, cross-modal attention fusion, and noise step-size-driven personalized optimization strategies.

[0011] In some embodiments, the method includes: Preprocess the input original electrocardiogram (ECG) signal to generate a short-time Fourier transform spectrogram, a Lorenz scatter plot, and the time-domain waveform of the original ECG signal. These three modal data are used as training set data. Construct a personalized arrhythmia classification model, including a diffusion-driven feature extractor, a cross-modal fusion module, and a classifier. Input each modal data into the diffusion model feature extractor to extract multi-scale robust features. Dynamically integrate the features of the three modalities through the cross-modal attention fusion module, and output the arrhythmia probability through the classifier with the integrated features. Freeze the parameters of the diffusion-driven feature extractor. Input the personalized optimized training set data into the personalized arrhythmia classification model, and train the parameters of the feature fusion module and the classifier part to obtain the trained personalized arrhythmia classification model. Collect the ECG signal to be detected, and preprocess it into a time-domain waveform, a short-time Fourier transform spectrogram, and a Lorenz scatter Figure 3 plot of three modalities, and perform personalized optimization. Input the optimized data into the trained personalized arrhythmia classification model to obtain the classification result.

[0012] Preferably, the diffusion-driven feature extractor adopts the downsampling encoding part and the middle bottleneck layer part in the pre-trained U-Net network. Specifically, the downsampling encoding part includes a convolutional layer and three levels of downsampling modules. Each level of downsampling module consists of two residual network blocks connected in series. The middle bottleneck layer includes two residual network blocks and a multi-head self-attention module. The self-attention mechanism of the multi-head self-attention module models the global dependency through query-key-value interaction.

[0013] Preferably, the pre-training method of the U-Net network is as follows: Input the noisy training set into the U-Net network. For each modality, perform pre-training through an unsupervised paradigm, using the KL divergence between the predicted noise and the real noise as the loss function. The method of adding noise to the training set data is as follows: , , where, represents the input original data, represents the noise randomly sampled from the standard normal distribution, represents an attenuation coefficient at time step , represents the noise intensity at each time step.

[0014] Preferably, the specific implementation of the cross-modal attention fusion module is as follows: After the time-domain waveform modality, short-time Fourier transform spectrogram modality, and Lorenz scatter plot modality data are independently encoded, the feature of each modality generates a query vector, a key vector, and a value vector respectively; The weights of each modality are obtained through query-key interaction calculation and then normalization; the features are fused according to the weights of each modality.

[0015] Preferably, the personalized optimization method adopts Bayesian optimization to select a noise-adding step size for each modality data; The personalized optimization method includes: Define the parameter set, parameter space, and optimization rounds of the noise-adding step size for each modality; Use Gaussian process to construct a surrogate function; Randomly initialize the parameter set as the initial observation point; Calculate the optimization index corresponding to the observation point, and update the surrogate function according to all known observation points; Based on the current surrogate function, use the upper confidence bound strategy to construct an acquisition function, select the next observation point according to the acquisition function, and observe the optimization index of this observation point; Iterate the above process until the preset optimization rounds are reached, and output the optimal parameter set.

[0016] Preferably, calculating the optimization index corresponding to the observation point and updating the surrogate function according to all known observation points are specifically as follows: Fine-tune the personalized arrhythmia classification model twice for the current parameter set, calculate the loss values of the two fine-tuning rounds based on the weighted cross-entropy loss function, and generate the optimization index value in combination with the model weight parameters , and the specific formula is as follows: , where, represents the loss function value of the model in the th round, represents the loss function value of the model in the th round, represents the weight parameter of the model in the th round; Update the surrogate function according to all known observation points; After the update, reset the personalized arrhythmia classification model to the initial state.

[0017] Preferably, the training process of the personalized arrhythmia classification model is as follows: Freeze the parameters of the feature extractor driven by diffusion for each modality; Use a personalized optimization method to add noise to each modality data, and then send it to a modality diffusion-driven feature extractor for feature extraction; Input the extracted features into a modality attention fusion module for fusion, and input the fused features into a classifier. Train a personalized arrhythmia classification model using weighted cross-entropy loss. The weighted cross-entropy loss is as follows: , where is the inverse frequency weight of class , represents the one-hot encoding of the true label of class , represents the probability that the model predicts this data belongs to class , represents the total number of arrhythmia classes.

[0018] Preferably, the electrocardiogram signal to be detected is first segmented, each segment has a length of 10 seconds, an overlap rate of 50%, and a sampling rate of 250 Hz.

[0019] In some embodiments, the device includes a processor and a memory storing program instructions. The processor is configured to execute the arrhythmia classification method based on a multi-modal diffusion model when running the program instructions.

[0020] A computer-readable storage medium stores a computer program, and when the program is executed by a processor, it implements the arrhythmia classification method based on a multi-modal diffusion model.

[0021] The arrhythmia classification method, device, and medium based on a multi-modal diffusion model provided by the embodiments of the present disclosure can achieve the following technical effects: (1) A feature extraction architecture based on a multi-modal diffusion model is proposed. Using the unsupervised training paradigm of the diffusion model, the noise components and multi-scale temporal features in the ECG signal are separated through a hierarchical denoising process, constructing a robust heart rhythm representation while reducing the dependence on data annotation.

[0022] (2) A cross-modal attention fusion module is designed to jointly utilize multi-source features of time-domain waveforms, spectrograms, and Lorenz scatter plots, achieving dynamic weight allocation and collaborative optimization of the feature space.

[0023] (3) A noise step-driven personalized optimization strategy is constructed, and the noise intensity of the input data is dynamically adjusted through a zero-order optimization method to adapt to the differential expression of individual ECG signals.

[0024] (4) Optimize the feature extraction network structure by using the neural network architecture search technology, remove redundant information paths and retain multi-granularity feature abstraction layers to improve the model calculation efficiency and generalization ability.

[0025] The above general description and the following description are only exemplary and explanatory, and are not used to limit this application. Brief Description of the Drawings

[0026] One or more embodiments are exemplarily illustrated by corresponding drawings. These exemplary illustrations and drawings do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements. The drawings do not constitute a scale limitation, and among them: Figure 1 is a schematic diagram of the method flow of the present invention; Figure 2 is a schematic diagram of the overall architecture of the method of the present invention; Figure 3 is a schematic diagram of the U-Net structure provided by the present invention; Figure 4 is a schematic diagram of the conversion of the ECG signal-Lorenz scatter plot of the present invention; Figure 5 is a schematic diagram of the surrogate function and the acquisition function of the Bayesian optimization process of the present invention; Figure 6 is a schematic diagram of the device structure provided by the present invention. Detailed Embodiments

[0027] In order to be able to understand the features and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure will be described in detail below in conjunction with the drawings. The attached drawings are only for reference and explanation, and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of explanation, multiple details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be shown in a simplified manner.

[0028] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so as to implement the embodiments of the present disclosure described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0029] Unless otherwise specified, the term "plurality" means two or more.

[0030] In the embodiments of the present disclosure, the character " / " indicates an "or" relationship between the preceding and following objects. For example, A / B means: A or B.

[0031] The term "and / or" is a description of the association relationship of an object, indicating that three relationships can exist. For example, A and / or B means: A or B, or, the three relationships of A and B.

[0032] The term "corresponding" may refer to an association relationship or a binding relationship. A corresponding to B means that there is an association relationship or a binding relationship between A and B.

[0033] Embodiment 1 As Figure 1 shown, an arrhythmia classification method based on a multimodal diffusion model realizes automatic judgment of arrhythmia with high robustness and low annotation cost by constructing a hierarchical feature extraction network, a multimodal dynamic fusion mechanism, and a personalized optimization strategy driven by a noise step size.

[0034] It includes: S1: Preprocess the input original electrocardiogram (ECG) signal to generate a short-time Fourier transform spectrogram and a Lorenz scatter plot, and use the three-modal data of the time-domain waveform of the original ECG signal as the training set data.

[0035] S2: Construct a personalized arrhythmia classification model, including a diffusion-driven feature extractor, a cross-modal fusion module, and a classifier; input each modal data into the diffusion model feature extractor for hierarchical denoising processing to extract multi-scale robust features, dynamically integrate the features of the three modalities through the cross-modal attention fusion module, and output the arrhythmia probability through the classifier with the integrated features.

[0036] S3: Freeze the parameters of the diffusion-driven feature extractor, input the personalized optimized training set data into the personalized arrhythmia classification model, and train the parameters of the feature fusion module and the classifier part to obtain the trained personalized arrhythmia classification model.

[0037] S4: Collect the electrocardiogram signal to be detected, preprocess it into three modalities of time-domain waveform, short-time Fourier transform spectrogram, and Lorenz scatter Figure 3 plot, and perform personalized optimization.

[0038] S5: Input the optimized data into the trained personalized arrhythmia classification model to obtain the classification result.

[0039] As a refinement of the above embodiment, as Figure 2 shown, the method of this embodiment includes three stages: multimodal data preprocessing, personalized arrhythmia classification model construction, and model prediction.

[0040] The above-mentioned multi-modal data preprocessing is involved in both the training and prediction of the model. The original ECG signals all need to go through the multi-modal data conversion module to generate the short-time Fourier transform spectrogram and Lorenz scatter plot modalities. Specifically, it includes: (1) Short-time Fourier transform spectrogram modality: This method uses the short-time Fourier transform to convert the original ECG signal into the frequency domain modality, and its process can be described by the following formula: , where, is a two-dimensional function defined in time and frequency, is the input original ECG segment, is the complex exponential function, representing the basis function of the Fourier transform, which is used to convert the time-domain signal to the frequency domain, is the angular frequency, is the center position of the current window, is the imaginary unit, represents the short-time Fourier transform operation, is the window function used to reduce the spectral energy leakage. The Hanning window is a commonly used window function, and its definition formula is as follows: , where, is the number of sampling points, is the window length.

[0041] (2) Lorenz scatter plot modality: The Lorenz scatter plot is a commonly used two-dimensional scatter plot in the field of cardiology, which shows the relationship between consecutive R-R intervals in the electrocardiogram. The coordinates of each point in the Lorenz scatter plot are calculated using the following formula: , where, is defined as a series of RR intervals by locating all the R peaks in the ECG segment. Referring to Figure 5 , is 0.8 - 0.5 = 0.3, is 0.5 - 1.0 = -0.5, then the coordinates of this point are (0.3, -0.5).

[0042] The personalized arrhythmia classification model consists of a diffusion-driven feature extractor, a cross-modal fusion module, and a personalized classifier. After being preprocessed by the multi-modal data preprocessing module, the input ECG signal synchronously generates the time-domain waveform, short-time Fourier transform (STFT) spectrogram, and Lorenz scatter Figure 3Modal data. Each modal data is respectively input into each modal diffusion feature extractor for noise injection and denoising processing, and robust representations are obtained through hierarchical feature extraction; the cross-modal fusion module uses an attention mechanism to dynamically integrate multi-source features, and finally the arrhythmia category probability is output by a lightweight classifier.

[0043] In the personalized arrhythmia classification model, the denoising network of each modal diffusion model uses a U-Net structure, which includes a symmetric downsampling encoding path, an intermediate bottleneck layer, and an upsampling decoding path. The feature maps of the same size between the encoding path and the decoding path are connected by a shortcut structure. Specifically, the input modality enters the encoding path through an initial one-dimensional convolutional layer (the number of channels is expanded to 48), and the feature map size is gradually compressed and the channel dimension is increased through three levels of downsampling modules: the first level maintains 48 channels, the second level is increased to 96, and the third level maintains 96 channels. Each level of the downsampling module is composed of two residual network blocks in series. Each residual block integrates weight-normalized convolution (convolution kernel size 3, stride 1), group normalization (number of groups 8), and SiLU activation function, and dynamically adjusts feature scaling and offset through a time embedding vector; at the end of each level, the feature map size is halved through a convolution with a stride of 2 (except for the last level). The intermediate bottleneck layer contains two residual network blocks and a multi-head self-attention module, and the number of channels remains 96. Among them, the self-attention mechanism models global dependencies through query-key value interaction, enhancing the ability to capture long-range patterns of heart rhythms. The decoding path uses three levels of upsampling modules. At each level, the feature map size is restored through transposed convolution and the number of channels is gradually reduced (96→48). At the same time, the feature maps of the corresponding levels of the encoding path are concatenated in channels, and multi-scale information is fused through residual blocks and linear attention modules. The final output layer aggregates shallow and deep layer features through residual blocks, and the number of channels is mapped to the target dimension by 1×1 convolution.

[0044] Since the U-Net network trained for the generation task introduces a mismatch between high-frequency noise and semantic abstraction, it may lead to a performance decline in the classification task. For the diffusion-driven feature extractor in the proposed method, the downsampling encoding part and the intermediate bottleneck layer part of the pre-trained denoising U-Net network are used. These modules capture the most abstract and high-level features from the input data while consolidating rich hierarchical features.

[0045] The specific implementation of the cross-modal attention fusion module is as follows: A cross-modal attention fusion module is designed for the heterogeneous characteristics of multi-modal data. After the time-domain, frequency-domain, and morphological features are independently encoded, query vectors , key vectors , and value vectors are respectively generated. The cross-modal weights are calculated through query-key interaction: For modality , its weight : , wherein, represents the feature dimension of the query vector and the key vector.

[0046] The contribution degree of each modality is obtained after normalization.

[0047] The final fused feature is mapped to a 128-dimensional high-level representation through a fully connected layer. This mechanism enables the model to adaptively focus on key modalities. For example, in atrial fibrillation detection, it strengthens the periodic anomaly features of spectrograms, and in ventricular premature beat classification, it emphasizes the QRS distortion of time-domain waveforms.

[0048] As a refinement of the above embodiment, in the pre-training stage of the U-Net network, a large-scale unlabeled ECG data is used. First, through a data processing module, the original unlabeled ECG data set is converted to generate a corresponding short-time Fourier transform spectrogram and Lorenz scatter plot modality data set. Then, for each modality, through an unsupervised paradigm, a denoising diffusion probability model is trained with minimizing the loss function as the optimization objective. The minimizing loss function is as follows: , wherein, is the noise randomly sampled from the standard normal distribution, represents the noisy data at time step t, represents the expectation of the following variables: time step t, true data sample , noise epsilon; is the model parameter, is the predicted output of the model.

[0049] Specifically, for each input training data, different degrees of noise addition are randomly performed, and the degree of noise addition is controlled by the noise addition step . The noise addition function is defined as follows: , , wherein, represents the original input data, represents the noise randomly sampled from the standard normal distribution, represents a decay coefficient at time step t, represents the noise intensity at each time step.

[0050] After that, the denoising network in the diffusion model predicts the added noise and calculates the KL divergence (Kullback-Leibler Divergence) between the predicted noise and the true noise, which is used as the loss function to optimize the denoising network. After the modal denoising diffusion probability model is trained, a partial structure of the denoising network is extracted as the diffusion-driven feature extractor.

[0051] As a refinement of the above embodiment, to adapt to individual ECG differences, a personalized optimization method driven by noise step size based on Bayesian optimization is proposed to select appropriate data noise addition step sizes for each modality. Define a personalized parameter set , which controls the noise injection intensity of each modality respectively; represents the time-domain waveform modality of the original ECG signal plus added noise, represents the short-time Fourier transform spectrogram modality plus added noise, represents the Lorenz scatter plot modality plus added noise. In order to evaluate the model convergence ability with as few training rounds as possible, an original optimization metric is proposed to characterize the loss decline efficiency of the model in the initial training stage, which is defined as follows: , where, represents the loss function value of the model in the -th round, represents the loss function value of the model in the -th round, represents the weight parameter of the model in the -th round; Select to use Gaussian Process to construct a surrogate function to approximate the objective function. Gaussian Process is a class of stochastic processes , and its any n-dimensional distribution follows a multivariate normal distribution, that is: for any finite number of , of any linear combination is a normal distribution. Just as a normal distribution can be determined by specifying the mean and variance, a Gaussian Process can be uniquely determined by specifying the mean function and the covariance function . In this method, the objective function is the mapping from the parameter set t to the proposed original optimization objective , denoted as , and the acquisition function constructed by Gaussian Process is defined as follows: , where, is the mean of the Gaussian function, set to the constant 0, For the covariance function, this method selects the radial basis function as the covariance function, and its definition is as follows:

[0052] where, is the length scale, and the larger its value, the stronger the exploration ability. indicates that the covariance function is used as the radial basis function.

[0053] The process of using the Gaussian process to construct the surrogate function is actually to use the Maximum A Posteriori estimate (MAP). Given the objective function at the known observation points, the function values of other parameter combinations in the parameter space are estimated. The process is defined as follows:

[0054]

[0055] where, represents the parameter combination, ) represents the probability distribution of the parameter combination under the actual observed values of the obtained objective function. After conversion using Bayes' formula, it is: , where, is the probability of obtaining the observed value given the parameters, is the prior probability of the parameter values. In this method, assuming that all parameter combinations have an equal probability distribution, the above MPA degenerates into the Maximum Likelihood Estimate (MLE). The surrogate function constructed using the Gaussian function essentially reflects the distribution probability of the value corresponding to each parameter combination (for reference, see Figure 5 ). The distribution probability of the value corresponding to each parameter combination at the function point follows a Gaussian distribution and only converges at the observed points (because there are actual objective function values after observation).

[0056] In the Bayesian optimization process, except for the initial observation points, the subsequent observation points are determined by the acquisition function. The acquisition function is generally established based on the current surrogate function. This method uses the upper confidence bound strategy to construct the acquisition function (as shown in Figure 5 ), that is, select the maximum value corresponding to each parameter combination estimated by the surrogate function to construct the function , and its definition is as follows: , where, is the covariance matrix of the Gaussian process, defined as .

[0057] Specifically, the personalized optimization method includes: (1) Defining the noise addition step size parameter sets, parameter spaces, and optimization rounds for each modality; and the parameter spaces and optimization rounds; (2) Using Gaussian processes to construct surrogate functions; (3) Randomly initializing the parameter sets as the initial observation points; (4) Calculating the optimization metrics corresponding to the observation points and updating the surrogate function based on all known observation points; (5) Based on the current surrogate function, constructing an acquisition function using the upper confidence bound strategy, selecting the next observation point according to the acquisition function, and observing the optimization metrics for this observation point; (6) Iterating the process of (4)-(5) until the preset optimization rounds are reached, and outputting the optimal parameter set.

[0058] The above process is consistent with the training process of the personalized arrhythmia classification model, but is only used to evaluate the training effect of the model under the current parameter set settings, and only two rounds of fine-tuning are performed, and the loss function values and model weight parameters obtained during the fine-tuning process are used to calculate the optimization objective , and the loss function uses the weighted cross-entropy function, which is consistent with the fine-tuning stage. After the evaluation of the current observation point, the updated model is not saved, and each observation during the optimization process starts from the model initialization state.

[0059] This process only requires 2 iterations to effectively evaluate the model performance under the current parameter settings, reducing the computational cost by more than 90% compared to the traditional objective.

[0060] As a refinement of the above embodiment, during the training process of the personalized arrhythmia classification model, the parameters of the diffusion-driven feature extractor for each modality are frozen, and only the parameters of the feature fusion module and the classifier part are trained.

[0061] Specifically as follows: Freezing the parameters of the diffusion-driven feature extractor for each modality; Adding noise to the data of each modality using the personalized optimization method, and then sending it into the diffusion-driven feature extractor for each modality to extract features; Inputting the extracted features into the modality attention fusion module for fusion, and inputting the fused features into the classifier (with a hidden layer dimension of 64 and ReLU activation), and training the personalized arrhythmia classification model using the weighted cross-entropy loss. The weighted cross-entropy loss is as follows: , where is the inverse frequency weight of class , represents class One-hot encoding of the true label, indicating the probability that the data predicted by the model belongs to the category of indicating the total number of arrhythmia categories. Weighted cross-entropy loss is used to overcome the imbalance of samples.

[0062] During the use phase, the electrocardiogram signals to be detected are first segmented, each segment is 10 seconds long, the overlap rate is 50%, and the sampling rate is 250Hz. After multi-modal conversion, diffusion feature extraction and fusion of each segment, the probabilities corresponding to each arrhythmia type of the ECG segment are output.

[0063] It should be noted that: the interference of false negative samples is eliminated through the multi-scale noise prediction mechanism of the diffusion model, the label dependence is reduced, and the feature robustness is improved; the dynamic collaboration of time-frequency-morphology features is realized by using cross-modal attention, enhancing the representation ability of complex heart rhythms; the relationship between the noise addition step size t and the model personalization ability is explored, and a noise step size optimization strategy based on convergence sensitivity is proposed, replacing the full model fine-tuning with zero-order search, and greatly reducing the personalization adaptation cost. Experiments show that this method improves the F1 value by 37.9% in cross-dataset testing and the personalization classification efficiency by 23.9%, providing reliable technical support for the accurate judgment of arrhythmia.

[0064] Next, in combination with the algorithm pseudo-code, the detailed implementation process of the pre-training of the U-Net network of the present invention, the personalization optimization method, and the training of the personalized arrhythmia classification model will be further described in detail.

[0065] Pre-training stage of the U-Net network The pre-training stage of the present invention follows the unsupervised training paradigm of the general denoising diffusion probability model, trains the diffusion model for the three modal data respectively, and extracts the parameters of the denoising network part after the training is completed.

[0066] Input: Initial diffusion model denoising network weights , unlabeled general data , maximum noise addition step size T, pre-training rounds EpochD.

[0067] Output: Feature extractor

[0068] For each modality

[0069] #T is set to 1000

[0070] For e in EpochsD:

[0071] end return

[0072] end Personalized Optimization Method Phase The step-driven personalized optimization strategy proposed in the present invention is based on the Bayesian optimization algorithm, uses an original optimization objective to evaluate model performance, and the labeled data used in the optimization process does not exceed 1000 pieces. During this process, the personalized classification model proposed by the method will not update its parameters.

[0073] Input: Parameter set , maximum number of iterations N, surrogate model set , acquisition function set , diffusion-driven feature extractor , classifier weight parameter

[0074] Output: Optimal parameter set

[0075] 1: Randomly initialize the parameter set

[0076] 2: for step in range(2) 3:

[0077] 4:

[0078] 5:

[0079] 6:

[0080] 7: Repeat 3 - 4 8:

[0081] 9: return

[0082] 10: while

[0083] 11: Based on the currently obtained point set Construct a surrogate model

[0084] 12: Based on the surrogate model , maximize the acquisition function Obtain the next evaluation point: 13:

[0085] 14: Repeat 2:9 15: end 16: return

[0086] The above algorithm's noise addition function is designed as follows, where is a decay coefficient at time step and is calculated through to obtain.

[0087] Personalized arrhythmia classification model training stage For each patient, the present invention uses no more than 300 labeled data for model fine-tuning to further improve the detection accuracy of the model on individual data. During the fine-tuning process, only the parameters of the classifier part are updated, and the parameters of the diffusion-driven feature extractor part are frozen, reducing the training cost.

[0088] Input: Optimal parameter set , maximum number of fine-tuning times EpochFT, diffusion-driven feature extractor , classifier weight parameters .

[0089] Output: Fine-tuned classifier weight parameters

[0090] for step in EpochFT # EpochFT is set to 20

[0091]

[0092]

[0093] end return

[0094] Combine Figure 6As shown in the figure, an arrhythmia classification device 300 based on a multi-modal diffusion model provided by an embodiment of the present disclosure includes a processor 304 and a memory 301. Optionally, the device may further include a communication interface 302 and a bus 303. Among them, the processor 304, the communication interface 302, and the memory 301 can communicate with each other through the bus 303. The communication interface 302 can be used for information transmission. The processor 304 can call the logical instructions in the memory 301 to execute the arrhythmia classification method based on the multi-modal diffusion model in the above embodiment.

[0095] In addition, when the logical instructions in the above-mentioned memory 301 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0096] The memory 301, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 304 executes functional applications and data processing by running the program instructions / modules stored in the memory 301, that is, implements the arrhythmia classification method based on the multi-modal diffusion model in the above embodiment.

[0097] The memory 301 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 301 may include a high-speed random access memory and may also include a non-volatile memory.

[0098] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, and the computer-executable instructions are set to execute the above-mentioned arrhythmia classification method based on the multi-modal diffusion model.

[0099] The above-mentioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transient computer-readable storage medium.

[0100] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for classifying arrhythmias based on a multimodal diffusion model, characterized in that: The following steps are involved: Preprocess the input original ECG signal to generate short-time Fourier transform spectrum and Lorentz scatter plot, and three modal data of the time domain waveform of the original ECG signal as training set data; A personalized arrhythmia classification model was constructed, including a diffusion-driven feature extractor, a cross-modal fusion module, and a classifier. The data of each modality was input into the diffusion model feature extractor to extract multi-scale robust features, and the features of the three modalities were dynamically integrated through the cross-modal attention fusion module. The integrated features were used to output the arrhythmia probability through the classifier. Freeze the diffusion-driven feature extractor parameters, input the training set data into the personalized arrhythmia classification model after personalized optimization, train the feature fusion module and some classifier parameters, and obtain the trained personalized arrhythmia classification model; Collect the ECG signals to be tested, pre-process them into three modes: time domain waveform, short-time Fourier transform spectrum and Lorentz scatter plot, and perform personalized optimization; The optimized data is input into the trained personalized arrhythmia classification model to obtain the classification results.

2. The arrhythmia classification method based on the multimodal diffusion model according to claim 1, characterized in that: The diffusion driven feature extractor uses the downsampling encoding part and the intermediate bottleneck layer part of the pre-trained U-Net network; Specifically, the downsampling encoding part includes a convolutional layer and a three-level downsampling module, and each level of the downsampling module is composed of two residual network blocks in series; the intermediate bottleneck layer includes two residual network blocks and a multi-head self-attention module, and the self-attention mechanism of the multi-head self-attention module models global dependencies through query-key-value interaction.

3. The arrhythmia classification method based on multimodal diffusion model according to claim 2, characterized in that: The U-Net network pre-training method is as follows: The training set is noisy and then fed into the U-Net network. For each modality, pre-training is performed using an unsupervised paradigm with the KL divergence between the predicted noise and the real noise as the loss function. The training set data is denoised in the following way: ， ， in, Represents the original input data, represents noise randomly sampled from a standard normal distribution, Indicates that at time step An attenuation coefficient when represents the noise intensity at each time step.

4. The arrhythmia classification method based on multimodal diffusion model according to claim 1, characterized in that: The specific implementation of the cross-modal attention fusion module is: After the time domain waveform modality, short-time Fourier transform spectrum modality and Lorentz scatter plot modality data are independently encoded, the query vector, key vector and value vector are generated for each modality feature; The weight of each modality is obtained through query-key interaction calculation and then normalization; the features are fused according to the weight of each modality.

5. The arrhythmia classification method based on multimodal diffusion model according to claim 1, characterized in that: The personalized optimization method adopts Bayesian optimization to select a noise adding step length for each modal data; The personalized optimization method comprises: Define the noise step parameter set, parameter space and optimization rounds for each mode; Use Gaussian processes to construct surrogate functions; Randomly initialize a set of parameters as the initial observation point; Calculate the optimization index corresponding to the observation point and update the proxy function based on all known observation points; Based on the current proxy function, a collection function is constructed using a confidence upper bound strategy, the next observation point is selected according to the collection function, and the optimization index observation is performed on the observation point; The above process is iterated until the preset optimization round is reached and the optimal parameter set is output.

6. The arrhythmia classification method based on multimodal diffusion model according to claim 5, characterized in that: Calculate the optimization index corresponding to the observation point and update the proxy function based on all known observation points as follows: Perform two rounds of personalized arrhythmia classification model fine-tuning on the current parameter set, calculate the loss value of the two rounds of fine-tuning based on the weighted cross entropy loss function, and generate the optimization index value in combination with the model weight parameters , the specific formula is as follows: ， in, Indicates that the model The loss function value of the round, Indicates that the model The loss function value of the round, Represents the model The weight parameter of the round; Update the proxy function based on all known observation points; After the update is complete, the personalized arrhythmia classification model is reset to the initialization state.

7. The arrhythmia classification method based on multimodal diffusion model according to claim 6, characterized in that: The personalized arrhythmia classification model training process is as follows: Freeze the parameters of each mode diffusion driven feature extractor; Use personalized optimization methods to add noise to each modal data, and then send it to the modal diffusion driven feature extractor for feature extraction; The extracted features are input into the modal attention fusion module for fusion, and the fused features are input into the classifier to train the personalized arrhythmia classification model using weighted cross entropy loss. as follows: ， in, For Category The inverse frequency weight of Indicates category The one-hot encoding of the true label, Indicates that the model predicts that this data belongs to the category The probability of Represents the total number of arrhythmia categories.

8. The arrhythmia classification method based on multimodal diffusion model according to claim 1, characterized in that: The ECG signal to be detected is first processed in segments, with each segment length of 10 seconds, an overlap rate of 50%, and a sampling rate of 250 Hz.

9. A device for classifying arrhythmias based on a multimodal diffusion model, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the arrhythmia classification method based on the multimodal diffusion model as described in any one of claims 1-8 when running the program instructions.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the arrhythmia classification method based on a multimodal diffusion model as described in any one of claims 1 to 8 is implemented.

Images