A method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram

Abstract

The application belongs to the technical field of medical signal processing, and in particular to a method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram, comprising the following steps: S1: data preprocessing of PCG and ECG original signals, including noise reduction, baseline drift elimination, power line interference elimination, etc., and normalization processing of the PCG and ECG signals, generation of power spectrum signals of heart sound and electrocardiogram by using the Welch method, and segmentation of the time domain signals and power spectrum signals of PCG and ECG into equal-length segments. In order to effectively solve the problem of inaccurate diagnosis caused by poor quality of single-mode data, a heart sound and electrocardiogram dynamic fusion module is constructed based on the time-frequency fusion results of the two signals, dynamic weight distribution is performed, and a suitable classifier is selected for classification, the frequency domain information is supplemented, and the weights are distributed according to the contributions of the heart sound and electrocardiogram signals, so that the detection and classification results of coronary heart disease are more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical signal processing, in particular to a method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram. BACKGROUND

[0002] Coronary heart disease is called "the world's number one killer", which is a global major public health challenge with concealment, stability, progression and fatality. The latest data from the China Disease Prevention and Control Center shows that there are about 11.39 million coronary heart disease patients in China, which is a high-incidence and fatal disease that endangers the health of the elderly. 28% of urban and rural residents died of coronary heart disease, and the social and economic burden was heavy. Coronary angiography, as the gold standard for detecting coronary heart disease, cannot be used as a routine detection method due to its invasiveness and high cost. Among them, the phonocardiogram (PCG) and electrocardiogram (ECG) signals are widely studied by domestic and foreign scholars because they reflect the electrical activity and mechanical state of the heart. It is found that they can effectively reveal the physiological state and abnormality of the heart. The combination of PCG and ECG provides more comprehensive information about the heart, and the two belong to different categories of signals and can complement each other. The information of PCG and ECG can improve the diagnosis of coronary heart disease and has a significant advantage in the diagnosis of coronary heart disease.

[0003] At present, the fusion of PCG and ECG signals in coronary heart disease classification has been widely concerned. The method of using PCG and ECG signals to diagnose coronary heart disease uses traditional hand-designed features and deep learning for classification, but it is time-consuming and laborious and the capture of features is not comprehensive enough. Using deep learning algorithm to extract PCG and ECG signal features and using machine learning algorithm for classification has become the mainstream research direction. However, the existing methods still have many problems: first, when the quality of a single signal is poor, if the weight of the signal cannot be dynamically adjusted, it may lead to inaccurate feature extraction. Specifically, a poor quality signal may be given too much weight, while a good quality signal may be given too little weight. This unreasonable weight distribution will lead to poor signal fusion results, thereby affecting the accuracy of the final classification result; second, the PCG and ECG signals of coronary heart disease patients often have increased low-frequency components and decreased high-frequency components in the frequency domain, but the existing deep learning technology fails to fully exploit the value of frequency domain information for coronary heart disease classification. Therefore, if the above problems can be effectively solved, the accuracy of coronary heart disease classification will be further improved, and the precision and reliability of the diagnosis result will be improved. SUMMARY

[0004] (I) Technical problems solved

[0005] In view of the deficiencies of the prior art, the present application provides a method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram, which solves the problems raised in the background art.

[0006] (II) Technical Solution

[0007] In order to achieve the above-mentioned purpose, the present application specifically adopts the following technical solution:

[0008] A method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram, comprising the following steps:

[0009] S1: data preprocessing is performed on the PCG and ECG original signals, including noise reduction, baseline drift elimination, power line interference elimination and the like, and normalization processing is performed on the PCG and ECG signals, a Welch method is used to generate power spectrum signals of the heart sound and the electrocardiogram, and the time domain signals and the power spectrum signals of the PCG and the ECG are segmented into equal-length segments;

[0010] S2: a PCG and ECG time-frequency feature extraction module with the same structure is constructed, a multi-scale convolution layer is introduced, which is used to capture shallow and deep features in time-frequency information, so as to generate spatial dimension features, a gated recurrent unit is used to capture long-term dependence in time-frequency information, so as to generate time dimension features, a global pooling layer and a convolution layer are introduced to further capture global features, a channel attention module is introduced to allocate weights to each channel, and a comprehensive space-time feature result is output through convolution;

[0011] S3: a PCG and ECG time-frequency fusion module with the same structure is constructed for joint processing and information supplement, position information is embedded into time domain and frequency domain features by using position encoding, a self-attention mechanism is used to extract and capture global dependence between features for the time domain signals and the power spectrum signals respectively, and a cross-attention mechanism is used to integrate frequency domain information as supplementary features into time domain features, so as to generate time-frequency fusion features, the fused features are further subjected to nonlinear transformation through a feedforward neural network, and residual connection and layer normalization are used to accelerate training and stabilize the optimization process of the network;

[0012] S4: PCG and ECG features are dynamically fused according to signal quality, a PCG and ECG dynamic fusion module with the same structure is constructed, the time-frequency fused features are weighted fused according to respective contributions, the module introduces a modal confidence, adjusts the weights of the PCG and the ECG dynamically, and reflects the influence of the respective on the final classification result, the module can adaptively optimize the feature fusion process, and accurate fused features are generated to input into a classifier to classify coronary heart disease.

[0013] Further, the step of generating power spectrum in S1 by using the Welch method is as follows: the heart sound and electrocardiogram signals are divided into N M-length sub-signals according to data length, a window function is added to each sub-signal to reduce the truncation effect of the signal, and discrete Fourier transform is performed on each windowed signal to obtain the frequency spectrum of each segment:

[0014]

[0015] wherein, is the power spectral estimate of the frequency segment;

[0016] The power spectral density of each frequency point is obtained by averaging the square of the amplitude of the Fourier transform result:

[0017]

[0018] wherein, is the power spectral estimate of the frequency segment, is a window function.

[0019] Further, the generated time domain signal and power spectrum signal in S2 are respectively input into a time-frequency feature extraction module, which includes five branches to extract time-space features. Three branches process the input signal through different sizes of convolution kernels to extract multi-scale features. One branch uses a global pooling operation to capture global feature information. One branch captures long-term dependencies in the signal sequence through a gated recurrent unit. The features of multiple channels are combined to achieve multi-scale time-space feature extraction, capturing patterns of different frequencies or time lengths. A ReLU layer and a BN layer are added after each convolution layer for nonlinear transformation and accelerated training. The channel attention module adjusts the weights of the channels through self-adaptive adjustment, highlighting key features and suppressing irrelevant features.

[0020] Further, the formula of the channel attention module is as follows:

[0021]

[0022] wherein, is the weight of the th channel, is the feature extraction matrix of the channel weight, is the channel feature extraction function, is an activation function;

[0023] After calculation by the channel attention module, the corresponding weight of each channel is obtained, and the formula is as follows:

[0024]

[0025] wherein, is the th channel feature processed by the channel attention mechanism, is the th channel feature processed by the channel attention mechanism.

[0026] Further, the time-frequency fusion module constructed in S3 introduces position encoding, and the encoded features and extracted features are concatenated and input into the self-attention module; the position encoding uses sine and cosine encoding to embed the sequence of time sequence information into the features, so that the model can utilize the sequence order information, and through the self-attention module, the parts of the features can be mutually concerned, so that the model can automatically focus on important features.

[0027] wherein the position encoding formula of sine and cosine is as follows:

[0028]

[0029] wherein, represents the position in the sequence, represents the index of the feature dimension, is the total feature dimension of the model, is a scaling factor that controls the sine and cosine waves of different frequencies to ensure the uniqueness of the encoding of different positions;

[0030] The position encoding feature and the feature extracted by the feature extraction module are superimposed to output a new feature, denoted as:

[0031]

[0032] wherein, is the output result of the time-frequency feature extraction module;

[0033] In the time-frequency fusion module, the time-domain features and the frequency-domain features with position encoding are input into the self-attention module, and the calculation formula is as follows:

[0034]

[0035] wherein, , , respectively represent the query, key and value of the feature; the function is used for normalization to ensure that the sum of the attention is 1;

[0036] The results obtained by the time-domain and frequency-domain through the self-attention mechanism are input into the cross-attention mechanism, the time-domain features are taken as the main features, and the frequency-domain information is integrated into the time-domain features as supplementary features, and the calculation formula is as follows:

[0037]

[0038] wherein, represents the key of the time-domain feature; , respectively represent the query and value of the time-domain feature.

[0039] Further, in the time-frequency fusion module in S3, the feedforward neural network is composed of two fully connected layers and a ReLU layer, which maps the features extracted by the attention mechanism to a higher-dimensional feature space and uses the ReLU layer to alleviate the problem of gradient disappearance; the skip connection is added in the residual connection, which makes the input directly added to the output, so that the model can retain the original features when learning new features, preventing the loss of feature information; the layer normalization adjusts the mean and variance of each layer of features to keep a stable distribution output.

[0040] Further, in the PCG and ECG dynamic fusion module in S4, based on the optimization principle of error upper bound, the relationship between the confidence and loss of PCG signal and ECG signal is used to dynamically adjust the weight, and the fusion weight of different modalities should be negatively correlated with the loss of the corresponding modality, i.e.

[0041]

[0042] wherein, represents the fusion weight of the modality , represents the loss of the corresponding modality;

[0043] The probability of predicting the coronary heart disease category is used as the weight of the fused signal:

[0044]

[0045] wherein, is the probability of predicting the coronary heart disease category, represents the weight of the modality ;

[0046] In the PCG and ECG dynamic fusion module, a relative calibration model is constructed to predict the fusion weight, a binary classification is selected for coronary heart disease, and the distribution uniformity of the binary classification is defined:

[0047]

[0048] wherein, represents the prediction probability of the modality for the category .

[0049] Further, in the PCG and ECG dynamic fusion module, considering the change of the environment, the uncertainty of ECG and PCG should be relative, and one modality should dynamically perceive the change of the other modality and modify its relative contribution to the system, and relative calibration is introduced to calibrate the relative uncertainty of each modality:

[0050]

[0051] wherein, and representing ECG and PCG, if is an ECG signal, then is a PCG signal;

[0052] According to the relative uncertainty of the modal, dynamically select to reduce or keep the modal weight, when <1, the modal has greater uncertainty, multiply the prediction accuracy and , reduce the contribution of the modal, when >1, the modal has smaller uncertainty and accurate prediction rate, keep the contribution of the modal, so the following asymmetric calibration term is defined:

[0053]

[0054] According to the asymmetric calibration term and the probability of the modal predicting the coronary heart disease category, the calibrated modal weight is obtained, and the formula is as follows:

[0055] .

[0056] Further, the PCG and ECG dynamic fusion module utilizes a relative calibration strategy to dynamically adjust the allocation weight , and then fuses the results of the PCG and ECG time-frequency features, and the dynamic fusion result is:

[0057]

[0058] wherein, and are the fusion weights of the ECG signal and the PCG signal and are the features after time-frequency fusion of the ECG signal and the PCG signal.

[0059] (Three) beneficial effects

[0060] Compared with the prior art, the present application provides a method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram, which has the following beneficial effects:

[0061] 1. This invention employs a time-frequency convolution method, cascading multi-scale convolution and gated recurrent units. First, convolutional layers with different kernel sizes, max-pooling layers, and gated recurrent units are run in parallel to extract spatiotemporal and global features at different time scales. Channel and spatial attention modules adaptively adjust the weights of different channels and spatial locations by combining channel and spatial attention. Finally, convolution further synthesizes the multi-scale spatiotemporal features. By employing multi-scale convolution and gated recurrent units, the model enhances its multi-scale feature capture capability, making it suitable for extracting features over long time periods, especially for processing long-sequence signals such as PCG and ECG.

[0062] 2. The cross-attention mechanism proposed in this invention integrates frequency domain features as supplementary information into time domain features and introduces positional encoding, enabling the fused time-frequency features to contain positional information. This not only perceives the absolute position in the sequence but also captures the relative relationships between positions, making it particularly effective in capturing long-distance dependencies when processing long-sequence signals such as PCG and ECG. Furthermore, the attention mechanism performs self-attention operations on features containing both positional and time-frequency information to further uncover the internal correlations of the features.

[0063] 3. This invention dynamically determines the weights of PCG and ECG signals through single-mode prediction rate and full-mode prediction rate, and employs a relative calibration strategy to dynamically adjust the weight of each mode based on the quality of other modes. The dynamic fusion module can adjust the mode weights according to the real-time quality of the data, thereby improving the reliability and stability of the fusion results under noisy conditions and reducing the risk of misjudgment caused by dependence on a single mode. Attached Figure Description

[0064] Fig. 1 This is a schematic diagram of the overall network framework of an embodiment of the present invention;

[0065] Fig. 2 This is a schematic diagram of the time-frequency feature extraction module according to an embodiment of the present invention;

[0066] Fig. 3 This is a schematic diagram of the time-frequency fusion module according to an embodiment of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Example

[0069] like Figs. 1-3As shown, one embodiment of the application proposes a method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram, comprising the following steps:

[0070] S1: data preprocessing is performed on the PCG and ECG original signals, including noise reduction, baseline drift elimination and power line interference elimination, etc. The PCG and ECG signals are normalized, the Welch method is used to generate the power spectrum signals of the heart sound and electrocardiogram, and the time domain signals and power spectrum signals of the PCG and ECG are segmented into equal-length segments.

[0071] Among them, the preprocessing part uses a 0.5-60Hz Butterworth band-pass filter to reduce noise and eliminate baseline drift of the ECG signal; a 25Hz high-pass Butterworth filter is used to eliminate low-frequency noise and baseline drift of the PCG signal; and a 50Hz IIR notch filter is used to eliminate power line interference in the ECG and PCG signals.

[0072] Among them, the steps of generating power spectrum by the Welch method are as follows: the heart sound and electrocardiogram signals are divided into N M-length sub-signals according to the data length, a window function is added to each sub-signal to reduce the truncation effect of the signal, and discrete Fourier transform is performed on each windowed signal to obtain the frequency spectrum of each segment:

[0073]

[0074] Among them, is the frequency domain representation of the first segment.

[0075] The amplitude square of the Fourier transform result is averaged to obtain the power spectrum density of each frequency point:

[0076]

[0077] Among them, is the power spectrum estimation of the frequency , is the window function.

[0078] S2: a PCG and ECG time-frequency feature extraction module with the same structure is constructed, a multi-scale convolution layer is introduced to capture shallow and deep features in the time-frequency information, thereby generating spatial dimension features. A gated recurrent unit is used to capture long-term dependencies in the time-frequency information, thereby generating time dimension features. A global pooling layer and a convolution layer are introduced to further capture global features. A channel attention module is introduced to allocate weights to each channel, and a comprehensive space-time feature result is output through convolution.

[0079] As Fig. 2As shown, the generated time-domain signal and power spectrum signal are input into the time-frequency feature extraction module, which contains five branches to extract spatiotemporal features. Three branches process the input signal through convolutional kernels of different sizes to extract multi-scale features; one branch uses global pooling to capture global feature information; and one branch uses a gated recurrent unit to capture long-term dependencies in the signal sequence. By combining features from multiple channels, multi-scale spatiotemporal feature extraction is achieved, capturing patterns of different frequencies or durations.

[0080] Specifically, the three one-dimensional convolutions have kernel sizes of 1, 3, and 5, with a stride of 1; the pooling layer has a kernel size of 3 and a stride of 1; the convolutional layer has a kernel size of 1 and a stride of 1; and the gated recurrent unit has 32 hidden units. Each convolutional layer is followed by a ReLU layer and a BN layer for non-linear transformation and accelerated training. The channel attention module adaptively adjusts the channel weights to highlight key features and suppress irrelevant features. Finally, a convolutional layer with a kernel size of 7 and a stride of 2 is introduced for feature dimensionality reduction to improve computational efficiency.

[0081] The formula for the channel attention module is as follows:

[0082]

[0083] in, It is the first The weight of each channel, It is the feature extraction matrix of channel weights. It is a channel feature extraction function. It is an activation function.

[0084] The weight of each channel is calculated using the channel attention module, as shown in the following formula:

[0085]

[0086] in, It is the first one after being processed by the channel attention mechanism. Channel characteristics, It is the first before the channel attention mechanism is processed. Each channel feature.

[0087] S3: Construct a PCG and ECG time-frequency fusion module with the same structure for joint processing and information supplement. Position information is embedded into the time domain and frequency domain features using position encoding. The self-attention mechanism is used to extract and capture the global dependence between features for time domain signals and power spectrum signals. The frequency domain information is integrated into the time domain features as supplementary features to generate time-frequency fusion features. The fused features are further nonlinearly transformed by a feedforward neural network, and residual connections and layer normalization are used to accelerate training and stabilize the optimization process of the network.

[0088] As shown in Fig. 3 , the obtained features are input into the constructed time-frequency fusion module. Position encoding is introduced, and the encoded features and extracted features are concatenated and input into the self-attention module. Specifically, the position encoding uses sine and cosine encoding to embed the sequence of time information into the features, allowing the model to utilize the sequential information of the sequence. Through the self-attention module, the parts of the feature can be mutually attentive, allowing the model to automatically focus on important features.

[0089] wherein the sine and cosine position encoding formula is as follows:

[0090]

[0091] wherein, represents the position in the sequence, represents the index of the feature dimension, is the total feature dimension of the model, is a scaling factor that controls the sine and cosine waves of different frequencies to ensure the uniqueness of the encoding of different positions.

[0092] The position encoding feature and the feature extraction module feature are superimposed to output a new feature, denoted as:

[0093]

[0094] wherein, is the output result of the time-frequency feature extraction module.

[0095] In the time-frequency fusion module, the time domain feature with position encoding and the frequency domain feature are passed through the self-attention module, and the following formula is used for calculation:

[0096]

[0097] wherein, , , represent the query, key, and value of the feature, respectively; the function is used for normalization to ensure that the sum of the attention is 1.

[0098] The results obtained by the self-attention mechanism in the time domain and the frequency domain are input into the cross-attention mechanism, the time domain features are taken as main features, and the frequency domain information is integrated into the time domain features as supplementary features, and the calculation formula is as follows:

[0099]

[0100] wherein, denotes the key of the time domain feature; , denote the query and value of the time domain feature, respectively.

[0101] In the PCG and ECG time-frequency fusion module, the feedforward neural network is composed of two fully connected layers and a ReLU layer, which maps the features extracted by the attention mechanism to a higher dimensional feature space, and uses the ReLU layer to alleviate the problem of gradient disappearance; the skip connection is added in the residual connection, so that the input is directly added to the output, so that the model can retain the original features when learning new features, preventing the loss of feature information; the layer normalization adjusts the mean and variance of each layer of features, so that it maintains a stable distribution output.

[0102] S4: dynamically fuse PCG and ECG features according to signal quality, construct PCG and ECG dynamic fusion modules with the same structure, and weight fuse the time-frequency fused features according to their respective contributions. This module introduces modal confidence, dynamically adjusts the weights of PCG and ECG, and reflects the influence of each on the final classification result. This module can adaptively optimize the feature fusion process, generate accurate fused features, and select an SVM classifier to classify coronary heart disease.

[0103] The optimization principle based on the upper limit of error in the PCG and ECG dynamic fusion module dynamically adjusts the weights using the relationship between the confidence and loss of PCG signals and ECG signals. The fusion weights of different modalities should be negatively correlated with the loss of the corresponding modalities, i.e.:

[0104]

[0105] wherein, denotes the fusion weight of the modality, denotes the loss of the corresponding modality. The probability of predicting the coronary heart disease class is used as the weight of the fused signal:

[0106]

[0107] wherein,

[0108] is the probability of predicting the coronary heart disease class, denotes the weight of the modality, .​

[0109] A relative calibration model is constructed in the PCG and ECG dynamic fusion module to predict the fusion weight. The present application selects to perform two classification on coronary heart disease, and therefore defines the distribution uniformity of two classification:

[0110]

[0111] wherein, represents the prediction probability of the modal to the class .

[0112] Considering the change of environment, in the PCG and ECG dynamic fusion module, the uncertainty of ECG and PCG should be relative. One modal should dynamically perceive the change of the other modal and modify its relative contribution to the system, and a relative calibration is introduced to calibrate the relative uncertainty of each modal:

[0113]

[0114] wherein, and represent ECG and PCG, if is the ECG signal, then is the PCG signal.

[0115] According to the relative uncertainty of the modal, the modal weight is dynamically selected to be reduced or kept , when <1, the modal has greater uncertainty, the prediction accuracy and are multiplied, and the contribution of the modal is reduced. >1, the modal has smaller uncertainty and accurate prediction rate, the contribution of the modal is kept, and therefore, the following asymmetric calibration term is defined:

[0116]

[0117] According to the asymmetric calibration term and the probability of the modal to predict the class of coronary heart disease, the calibrated modal weight is obtained, and the formula is as follows:

[0118]

[0119] The PCG and ECG dynamic fusion module dynamically adjusts the distribution weight by using the relative calibration strategy , and then fuses the results of PCG and ECG time-frequency features, and the dynamic fusion result is:

[0120]

[0121] wherein, and are the fusion weights of ECG signal and PCG signal respectively and are the features of ECG signal and PCG signal after time-frequency fusion respectively.

[0122] Finally, the fusion result is input into a classifier for coronary heart disease classification.

[0123] The present application provides a method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram, which can dynamically adjust the weights of PCG and ECG signals, especially suitable for scenarios where the quality of single modality signal is poor. The method also introduces position information by combining position coding and self-attention mechanism, focuses on internal correlation, and fully utilizes the frequency domain information of the signal. Through the cross-attention mechanism, the frequency domain features are integrated into the time domain features as supplementary information to improve the accuracy of subsequent classification. Therefore, the method proposed in this application can be used as an effective and feasible method for classifying coronary heart disease.

[0124] Finally, it should be noted that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram, characterized in that: Includes the following steps: S1: Perform data preprocessing on the raw PCG and ECG signals, including noise reduction, baseline drift elimination and power line interference elimination, and normalize the PCG and ECG signals. Use the Welch method to generate power spectrum signals of heart sounds and ECG, and divide the time domain signals and power spectrum signals of PCG and ECG into equal-length segments. S2: Construct PCG and ECG time-frequency feature extraction modules with the same structure, introduce multi-scale convolutional layers to capture shallow and deep features in time-frequency information, thereby generating spatial dimension features. Use gated recurrent units to capture long-term dependencies in time-frequency information to generate temporal dimension features. Introduce global pooling layers and convolutional layers to further capture global features. Introduce channel attention modules to allocate weights to each channel and output comprehensive spatiotemporal feature results through convolution. S3: Construct PCG and ECG time-frequency fusion modules with the same structure for joint processing and information supplementation. Position encoding is used to embed positional information into time-domain and frequency-domain features. Self-attention mechanism is used to extract and capture global dependencies between features for time-domain signals and power spectrum signals respectively. Frequency-domain information is integrated into time-domain features as supplementary features through cross-attention mechanism to generate time-frequency fusion features. The fused features are further nonlinearly transformed through feedforward neural network. Residual connections and layer normalization are used to accelerate the training and stabilize the optimization process of the network. The constructed time-frequency fusion module introduces position encoding, and the encoded features and extracted features are concatenated and input into the self-attention module. Position encoding uses sine and cosine encoding to embed the sequence of time information into the features, so that the model can use the sequential information of the sequence. Through the self-attention module, the different parts of the feature itself pay attention to each other, so that the model can automatically focus on important features. S4: Based on the dynamic fusion of PCG and ECG features according to signal quality, a dynamic fusion module with the same structure for PCG and ECG is constructed. The time-frequency fused features are weighted and fused according to their respective contributions. This module introduces modal confidence and dynamically adjusts the weights of PCG and ECG to reflect their respective impact on the final classification result. This module can adaptively optimize the feature fusion process, generating accurate fused features to input into the classifier for classifying coronary heart disease. The PCG and ECG dynamic fusion module uses the optimization principle based on the upper limit of error, utilizing the relationship between the confidence of PCG and ECG signals and the loss to dynamically adjust the weights. The fusion weights of different modalities should be negatively correlated with the loss of the corresponding modality, i.e.: ; in, The mode is represented as The fusion weight, This represents the loss of the corresponding mode; The probability of predicting the type of coronary artery disease is used as the weight of a certain signal after fusion: ; in, To predict the probability of different types of coronary heart disease, The mode is represented as The weights; In the dynamic fusion module of PCG and ECG, a relative calibration model is constructed to predict the fusion weights. A binary classification of coronary artery disease is selected, and the uniformity of the binary classification distribution is defined. ; in, Representing modes Category The predicted probability.

2. The method for joint detection of coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram according to claim 1, characterized in that: The steps for generating the power spectrum using the Welch method in S1 are as follows: The heart sound and electrocardiogram signals are divided into N sub-signals of length M according to their data length. A window function is applied to each sub-signal to reduce signal truncation. A discrete Fourier transform is then performed on each windowed signal to obtain the spectrum of each segment. ; in, It is the first Frequency domain representation of the segment; The power spectral density at each frequency point is obtained by averaging the squared amplitudes of the Fourier transform results. ; in, It is frequency Power spectrum estimation, It is a window function.

3. The method for detecting coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram according to claim 1, characterized in that: In step S2, the generated time-domain signal and power spectrum signal are input into the time-frequency feature extraction module, which contains five branches to extract spatiotemporal features. Three of these branches process the input signal through convolutional kernels of different sizes to extract multi-scale features; one branch uses global pooling to capture global feature information. One branch captures long-term dependencies in the signal sequence through a gated recurrent unit; it combines features from multiple channels to extract multi-scale spatiotemporal features, capturing patterns of different frequencies or durations. ReLU and BN layers are added after each convolutional layer for nonlinear transformation and accelerated training. The channel attention module adjusts the channel weights adaptively to highlight key features and suppress irrelevant features.

4. The method for joint detection of coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram according to claim 3, characterized in that: The formula for the channel attention module is as follows: ; in, It is the first The weight of each channel, It is the feature extraction matrix of channel weights. It is a channel feature extraction function. It is an activation function; The weight of each channel is calculated using the channel attention module, as shown in the following formula: ; in, It is the first one after being processed by the channel attention mechanism. Channel characteristics, It is the first before the channel attention mechanism is processed. Each channel feature.

5. The method for joint detection of coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram according to claim 4, characterized in that: The position encoding formulas for sine and cosine in S3 are as follows: ; in, Indicates the position in the sequence. Index representing the feature dimension It is the total feature dimension of the model. It is a scaling factor that controls sine and cosine waves of different frequencies to ensure that the encoding at different positions is unique; The location encoding features and the features from the feature extraction module are superimposed to output a new feature, represented as: ; in, This is the output of the time-frequency feature extraction module; In the time-frequency fusion module, the time-domain features and frequency-domain features with location encoding are calculated using the following formula in the self-attention module: ; in, , , These represent the query, key, and value of the feature, respectively. The function is used for normalization, ensuring that the sum of the attention values ​​in the output is 1; The results obtained from the time and frequency domains through the self-attention mechanism are input into the cross-attention mechanism. The time domain features are used as the main features, and the frequency domain information is integrated into the time domain features as supplementary features. The calculation formula is as follows: ; in, Keys representing time-domain features; , These represent queries and values ​​related to time-domain features, respectively.

6. The method for joint detection of coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram according to claim 1, characterized in that: In the time-frequency fusion module of S3, the feedforward neural network consists of two fully connected layers and one ReLU layer. It maps the features extracted by the attention mechanism to a higher-dimensional feature space and uses the ReLU layer to alleviate the gradient vanishing problem. Skip connections are added to the residual connections so that the input is directly added to the output, so that the model can retain the original features when learning new features and prevent the loss of feature information. Layer normalization adjusts the mean and variance of the features of each layer to keep the output distribution stable.

7. The method for joint detection of coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram according to claim 1, characterized in that: In the aforementioned PCG and ECG dynamic fusion module, considering environmental changes, the uncertainties of ECG and PCG should be relative. One mode should dynamically sense changes in the other mode and modify its relative contribution to the system. Relative calibration is introduced to calibrate the relative uncertainty of each mode. ; in, and Representing ECG and PCG, if If it is an ECG signal, then PCG signal; Dynamically select to reduce or maintain based on the relative uncertainty of the mode. Modal weights, in <1 hour, Modalities have greater uncertainty, which will affect the accuracy of predictions and Multiply, decrease Modal contribution, in When >1, Modal analysis exhibits lower uncertainty and higher predictive accuracy, maintaining... The modal contribution is considered, therefore the following asymmetric calibration term is defined: ; The calibrated modal weights are obtained based on the asymmetric calibration term and the probability of modal prediction of coronary artery disease category, as shown in the following formula: 。 8. The method for joint detection of coronary heart disease based on time-frequency fusion of heart sound and electrocardiogram according to claim 7, characterized in that: The PCG and ECG dynamic fusion module dynamically adjusts the weight allocation using a relative calibration strategy. Then, the time-frequency feature results of PCG and ECG are fused together, and the dynamic fusion result is as follows: ; in, and The fusion weights for ECG and PCG signals respectively and These are the characteristics of the ECG and PCG signals after time-frequency fusion, respectively.

Images