Multi-mode heart and brain disease preliminary screening system based on manual feature guidance

Abstract

The invention discloses a multi-mode heart and brain disease preliminary screening system based on manual feature guidance, and the system comprises a data collection and processing module which is used for collecting and preprocessing multi-channel physiological signal data; the multi-granularity feature extraction module is used for performing coarse-granularity feature extraction on the physiological signal data, and forming table data by using the extracted features as a coarse-granularity manual feature matrix; meanwhile, physiological signal data passes through a deep convolutional network to form a fine-grained signal feature matrix; the feature fusion module is used for weighting the coarse-grained manual feature matrix to the fine-grained feature matrix in an attention form, and screening and removing redundant features with low importance to obtain fusion features; and the preliminary screening prediction module inputs the fusion features into a deep learning classification network, outputs a cardio-cerebral disease result, and preliminarily screens the cardio-cerebral disease occurrence risk. According to the invention, the accuracy of predicting the cardio-cerebral disease can be improved, and the preliminary screening of the cardio-cerebral disease occurrence risk can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of initial screening and prediction of cardio-cerebral diseases, and in particular to a multimodal initial screening system for cardio-cerebral diseases based on manual feature guidance. Background Art

[0002] There are many factors that affect cardiovascular and cerebrovascular diseases. Early and accurate diagnosis of CVD is crucial to promoting effective treatment and reducing the risk of related death. The types of data generated during the diagnosis process are rich and varied, including tabular data, physiological signal data, etc. Multi-modal data can assist in pre-screening and treatment, and improve the accuracy of single-modal data diagnosis.

[0003] However, traditional cardiovascular and cerebrovascular disease detection methods rely on manual operations by doctors and only utilize medical diagnostic data from a single modality, and information fusion is not intelligent enough.

[0004] A Chinese patent document with publication number CN114903445A discloses an intelligent monitoring and early warning system for cardiovascular and cerebrovascular diseases, comprising: a wearable device, an acquisition module, a transmission module, an intelligent terminal module and a display module; the wearable device is worn on the wrist of the person being tested, and the acquisition module, the transmission module and the display module are all arranged on the wearable device; the acquisition module is used to acquire the physiological signals of the person being tested; the transmission module is used to transmit the physiological signals to the intelligent terminal module, and is also used to transmit the data of the intelligent terminal module to the display module; the intelligent terminal module is used to process the physiological signals; the display module is used to provide early warning of cardiovascular and cerebrovascular disease risks and display predicted cardiovascular and cerebrovascular disease results.

[0005] A Chinese patent document with publication number CN116269391A discloses a heart-brain coupling analysis and evaluation method and system thereof, which collects brain oxygen signals and electrocardiogram signals corresponding to multiple test users in real time and synchronously, performs data preprocessing to obtain processed brain oxygen signals, processed electrocardiogram signals and cerebral blood oxygen variability; performs feature extraction on the processed electrocardiogram signals, respectively, and calculates to obtain an electrocardiogram feature set, a heart rate signal and a heart rate variability feature set; performs heart-brain coupling analysis and evaluation based on the processed brain oxygen signals and heart rate signals, and obtains a heart-brain coupling feature set; establishes a mental stress assessment model in combination with an integrated learning method, corrects the output mental stress value, and improves the accuracy and applicability of mental stress assessment.

[0006] The above patents only use traditional time domain, frequency domain features, and heart rate variability feature extraction methods, which often rely on manual design and may not be able to fully tap the deep information in the original signal, ignoring the extraction of local detail features. Deep convolution can automatically learn more robust feature representations and introduce the participation of knowledge in the field of heart and brain, which is particularly important for complex heart and brain signals.

[0007] Therefore, the clinical application of an intelligent method for initial screening of cardiovascular and cerebrovascular diseases is of great significance and can help patients and doctors prevent diseases at an early stage. Summary of the invention

[0008] The present invention provides a multimodal cardio-cerebral disease preliminary screening system based on manual feature guidance, which can improve the accuracy of predicting the incidence of cardio-cerebral diseases and realize preliminary screening of the risk of cardio-cerebral diseases.

[0009] A multimodal cardio-cerebral disease screening system based on manual feature guidance, including:

[0010] Data acquisition and processing module, used to collect multi-channel physiological signal data and perform pre-processing;

[0011] The multi-granularity feature extraction module is used to extract coarse-grained features from the preprocessed physiological signal data, and to form the extracted features into tabular data as a coarse-grained manual feature matrix; at the same time, the preprocessed physiological signal data is formed into a fine-grained signal feature matrix after passing through a deep convolutional network;

[0012] The feature fusion module is used to weight the coarse-grained manual feature matrix to the fine-grained feature matrix in the form of attention, filter out redundant features with low importance, and obtain fused features;

[0013] The initial screening prediction module inputs the fusion features into the deep learning classification network, outputs the results of cardiovascular and cerebrovascular diseases, and preliminarily screens the risks of cardiovascular and cerebrovascular diseases.

[0014] Furthermore, the physiological signal data includes electroencephalogram (EEG), electrocardiogram (ECG), blood oxygen saturation (SpO2), blood pressure (HP), and respiratory rate (RPF).

[0015] In the data acquisition and processing module, multi-channel physiological signal data is collected, specifically:

[0016] At a specific sampling frequency f c The signals of multiple physiological signal channels are collected synchronously and arranged in time sequence into a multidimensional data structure. Where c is the number of signal channels. To facilitate the representation of downsampled signals, n·t is used as the subscript in the timestamp sequence, n·t=1, 2, 3...T, where T represents the total number of samples. The sampling value of each physiological signal channel corresponds to a specific timestamp, and there is a correlation between the time series of each signal channel of the same subject.

[0017] Preprocessing includes denoising and downsampling; in denoising, high-pass filtering, low-pass filtering and wavelet processing are used to remove high-frequency noise, low-frequency interference and baseline drift in the signal; and downsampling method is used for the denoised signal to retain some sampling points in the original signal to reduce the sampling rate, and the downsampling rate is n.

[0018] In the multi-granularity feature extraction module, coarse-grained feature extraction is performed on the preprocessed physiological signal data, specifically:

[0019] EEG ECG signal Blood oxygen saturation blood pressure Respiratory rate Extract general frequency domain features F fre , general time domain feature F time , for ECG signals Extracted heart rate variability feature F hrv ; Common time domain features include maximum value, minimum value, mean value, standard deviation, variance, root mean square value, skewness, and kurtosis; Common frequency domain features include frequency mean, root mean square frequency, and standard deviation frequency; Heart rate variability features include RR interval standard deviation F extracted based on the time interval sequence between adjacent heart beats sdnn , RR interval root mean square F rmssd , the number of times the RR interval difference exceeds 50ms F nn50 And the percentage F pnn50 ;

[0020] The general frequency domain feature F fre , general time domain feature F time , heart rate variability characteristics F hrv Splice to form coarse-grained manual features

[0021] In the multi-granularity feature extraction module, the pre-processed physiological signal data is passed through a deep convolutional network to form a fine-grained signal feature matrix, specifically:

[0022] Preprocessed multi-channel physiological signals A multi-layer deep convolutional network is used for layer-by-layer convolution. Each layer contains a pair of convolutional layers C1(·), C2(·) and a residual connection layer R(·). The following formula is used to calculate the multi-channel physiological signals. Perform fine-grained feature extraction to obtain fine-grained features F sig

[0023]

[0024] Where l represents the number of convolutional layers in the deep network.

[0025] In the feature fusion module, the coarse-grained manual feature matrix is ​​weighted to the fine-grained feature matrix in the form of attention, specifically:

[0026] The manual feature F hand Through the fully connected function f fc (·) Feature projection, the manual feature after projection is f fc (F hand ) ; then calculate the weighted feature F guided by the coarse-grained manual features weight ; The weighted feature F weight and the projected manual feature f fc (F hand ) to obtain the fusion features

[0027] The formula for calculating the weighted features guided by coarse-grained manual features is as follows:

[0028] F weigh =softmax(f fc (F hand )·F sig T )·F sig

[0029] In the formula, F sig It is a fine-grained feature.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention can extract multi-granularity feature information from physiological signal monitoring data by using multiple data sources and innovative fusion guidance methods, including coarse-grained table manual features and fine-grained physiological signal features. The coarse-grained table features are fused into the signal features in a weighted attention manner, removing redundant information in long time series data and focusing on deep convolutional features that are consistent with the potential distribution of coarse-grained features, thereby improving the accuracy of prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a structural diagram of a multimodal cardio-cerebral disease screening system based on manual feature guidance according to the present invention;

[0033] Figure 2 It is a specific structural diagram of the multi-granularity feature extraction module in the present invention;

[0034] Figure 3 It is a specific structural diagram of the feature fusion module in the present invention;

[0035] Figure 4 It is a comparison diagram of prediction results in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

[0037] like Figure 1 As shown, a multimodal cardio-cerebral disease initial screening system based on manual feature guidance includes: a data acquisition and processing module, a multi-granularity feature extraction module, a feature fusion module and an initial screening prediction module.

[0038] The data acquisition and processing module is used to collect the original multi-channel physiological signal data. In a home environment or a hospital environment, multi-channel physiological monitoring signal data of a specific group is collected through auxiliary medical wearable devices. The multi-channel physiological monitoring signals include electroencephalogram (EEG), electrocardiogram (ECG), blood oxygen saturation (SpO2), blood pressure (HP), respiratory rate (RPF), etc. The multi-channel physiological signals collected at a high sampling rate are preprocessed, and the preprocessing methods include denoising and downsampling.

[0039] The multi-granularity feature extraction module is used to extract coarse-grained features from physiological signal data. The feature extraction is performed under the intervention of knowledge in the field of cardio-cerebral diseases, including universal frequency domain features, universal time domain features, and heart rate variability features extracted from electrocardiogram (EEG) signals. The extracted features form tabular data as a coarse-grained manual feature matrix. The physiological signal data is formed into a fine-grained signal feature matrix after a deep convolutional network.

[0040] The feature fusion module is used to fuse multi-modal and multi-granular feature matrices. The present invention adopts an advanced manual feature-guided multi-modal feature fusion technology, which weights the coarse-grained manual feature matrix to the fine-grained feature matrix in the form of attention, screens out redundant features with low importance, and obtains fused features.

[0041] The initial screening prediction module is used to predict and evaluate individual cardiovascular and cerebrovascular risk conditions and compare them with the actual categories. Through the combined features of multi-modal and multi-granularity feature fusion, the cardiovascular and cerebrovascular disease results are obtained to preliminarily screen the risk of cardiovascular and cerebrovascular diseases:

[0042] In the embodiment of the present invention, multi-channel physiological signal acquisition is performed at a specific sampling frequency f c The signals of multiple physiological signal channels are collected synchronously and arranged in time sequence into a multidimensional data structure. Where c is the number of signal channels. To facilitate the representation of downsampled signals, n·t is used as the subscript in the timestamp sequence, n·t = 1, 2, 3...T, T represents the total number of samples. The sampling value of each physiological signal channel corresponds to a specific timestamp, and the time series between the signal channels of the same subject are correlated.

[0043] Data preprocessing method is a method based on multi-channel physiological signals Based on the inherent properties of the signal, high-pass filtering, low-pass filtering, and wavelet denoising are used to remove high-frequency noise, low-frequency interference, and baseline drift in the signal. A comprehensive method of downsampling is used for the denoised signal. Some sampling points in the original signal are retained to reduce the sampling rate, and the downsampling rate is n. The c-dimensional physiological signal obtained after processing is expressed as

[0044] Due to multiple factors such as respiration, body movement, poor electrode contact, and skin electrode impedance, multi-channel physiological monitoring signals usually contain noise or unwanted artifacts, which may affect the analysis of the signal. Therefore, in the data acquisition and processing module, multiple signal denoising methods are used for preprocessing, including high-pass filtering, low-pass filtering, and wavelet denoising. This method not only removes baseline drift, power frequency interference, and high-frequency noise in the signal, but also retains the key features of the signal and avoids the loss of useful information. For the denoised signal, a downsampling operation is performed to reduce the unnecessary redundant information generated in the signal at a high sampling frequency.

[0045] Multi-granularity feature extraction module such as Figure 2 As shown in the figure, it is a feature extraction method for multi-channel physiological signal data under the guidance of knowledge in the field of cardiovascular and cerebrovascular diseases. According to the knowledge in the field of cardiovascular and cerebrovascular diseases, the heart rate variability characteristics of ECG signals include rich feature information related to the heart and brain. Specifically, the time interval between two adjacent R wave peaks represents a complete cardiac cycle of the heart, and its standard deviation, root mean square value, and heart rate jump value can all reflect the health of the heart. Coarse-grained feature extraction includes EEG signals. ECG signal Blood oxygen saturation blood pressure Respiratory rate Extract the general frequency domain feature F fre , general time domain feature F time , and the ECG signal The heart rate variability feature F extracted from hrv The signal time domain characteristics include the maximum value F max , minimum value F min , mean F mean , standard deviation F std , variance F var , RMS value Frms , skewness F ske , Kurtosis F kur The signal frequency domain characteristics include the frequency mean F avg , frequency root mean square F square , frequency standard deviation F fre_std The heart rate variability features include the RR interval standard deviation F extracted based on the time interval sequence between adjacent heart beats. sdnn , RR interval root mean square F rmssd , the number of times the RR interval difference exceeds 50ms F nn50 And the percentage F pnn50 . The following formulas are used for calculation:

[0046]

[0047]

[0048]

[0049] in, It's a signal The spectrum of k = 1, 2, ..., K, K is the number of spectral lines, f(k) is the frequency value of the kth spectral line, N is the total number of RR intervals in the ECG signal, and I(·) is the indicator function. When the condition in the brackets is met, the value is 1, otherwise it is 0. Time domain characteristics Frequency domain characteristics Heart rate variability characteristics

[0050] The time domain features, frequency domain features, and heart rate variability features are spliced ​​to form coarse-grained manual features. A manual feature matrix based on domain knowledge is constructed.

[0051] like Figure 2 Fine-grained signal feature extraction is shown in Figure 2. This is a method for preprocessing multi-channel physiological signals. The d-layer deep convolutional network layer-by-layer convolution method is adopted, which is composed of l basic component network layers stacked, each basic layer contains a pair of convolutional layers C1(·), C2(·) and a residual connection layer R(·). For signal time series. The signal is calculated by the following formula Perform fine-grained cross-time step feature extraction to obtain fine-grained features The fine-grained feature F sig The dimension is n×d, where n is the number of samples and d is the feature length after processing by the deep convolutional network.

[0052] Multidimensional physiological monitoring signals are a complex and changeable data structure that contains multi-level information. In this case, relying solely on feature information of a single granularity may not be sufficient to fully capture the effective information in the signal, especially in the early characterization of the disease. Therefore, the multi-granularity feature extraction module we proposed combines coarse-grained and fine-grained features extracted from multi-dimensional physiological monitoring signals to achieve a comprehensive characterization of different information levels in the signal. Manual coarse-grained features represent feature information that is highly relevant to the study of cardio-cerebral health, and fine-grained features are extracted through deep convolutional networks, which can capture complex features in the signal that are difficult to manually design, and achieve higher feature discrimination. The necessity of this multi-granularity feature extraction method is reflected in its ability to find a balance between refinement and integration, providing richer feature information for accurate diagnosis of diseases.

[0053] The weighted fusion method in the feature fusion module is as follows Figure 3 As shown in the figure, it is possible to balance the feature information at different levels between coarse-grained and fine-grained features, and propose a model for the classification of cardiovascular and cerebrovascular diseases using manual features guided by multi-granularity and multi-modal data. The core idea is to use the attention mechanism to affect the fine-grained, multi-channel physiological signal features with coarse-grained manual table features, and then fuse the weighted signal features guided by the manual table features with the manual table features. Disease classification is performed based on the features after fusion of multi-modal features. The manual feature F hand Through the fully connected function f fc (·) Feature projection, which is in the same feature space R as the signal feature n×d , n is the number of samples in the population, and d is the number of feature dimensions. Then calculate the weighted feature F after the coarse-grained manual feature guidance weight =softmax(f fc (F han )·F sig T )·F sig . The weighted features are combined with the projected manual features to obtain the fused features. First, the manual feature matrix, as a representative of domain knowledge, weighted selection of fine-grained features ensures that the model focuses on the key features of cardiovascular and cerebrovascular diseases and improves the discriminative power of features; secondly, through the attention mechanism, the model can automatically learn the weight distribution of different features, effectively enhancing the model's ability to capture potential correlations between multimodal data. The multimodal feature fusion model constructed in this way not only improves the accuracy of classification and recognition, but also enhances the robustness of the model, making it suitable for the analysis of complex physiological signals and the detection of disease states.

[0054] The initial screening prediction module will integrate the features through a deep learning classification network to screen the individual's cardiovascular and cerebrovascular diseases. It will determine whether there is a risk of cardiovascular and cerebrovascular diseases based on the individual classification results, thereby achieving early screening.

[0055] like Figure 4 As shown in FIG. 1 , the proposed method is compared with two other single-modal methods, and their accuracy indicators are evaluated on the same data set. The proposed method achieves better performance, which proves the beneficial effects of the present invention.

[0056] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multimodal cardio-cerebral disease screening system based on manual feature guidance, characterized in that: include: Data acquisition and processing module, used to collect multi-channel physiological signal data and perform pre-processing; The multi-granularity feature extraction module is used to extract coarse-grained features from the preprocessed physiological signal data, and to form the extracted features into tabular data as a coarse-grained manual feature matrix; at the same time, the preprocessed physiological signal data is formed into a fine-grained signal feature matrix after passing through a deep convolutional network; The feature fusion module is used to weight the coarse-grained manual feature matrix to the fine-grained feature matrix in the form of attention, filter out redundant features with low importance, and obtain fused features; The initial screening prediction module inputs the fusion features into the deep learning classification network, outputs the results of cardiovascular and cerebrovascular diseases, and preliminarily screens the risks of cardiovascular and cerebrovascular diseases.

2. The multimodal cardio-cerebral disease screening system based on manual feature guidance according to claim 1 is characterized in that: Physiological signal data includes EEG signals, ECG signals, blood oxygen saturation, blood pressure, and respiratory rate.

3. The multimodal cardio-cerebral disease screening system based on manual feature guidance according to claim 1 is characterized in that: In the data acquisition and processing module, multi-channel physiological signal data is collected, specifically: At a specific sampling frequency f c The signals of multiple physiological signal channels are collected synchronously and arranged in time sequence into a multidimensional data structure. Where c is the number of signal channels. To facilitate the representation of downsampled signals, n·t is used as the subscript in the timestamp sequence, n·t=1, 2, 3...T, where T represents the total number of samples. The sampling value of each physiological signal channel corresponds to a specific timestamp, and there is a correlation between the time series of each signal channel of the same subject.

4. The multimodal cardio-cerebral disease screening system based on manual feature guidance according to claim 1 is characterized in that: In the data acquisition and processing module, preprocessing includes denoising and downsampling; Among them, high-pass filtering, low-pass filtering and wavelet processing are used for denoising to remove high-frequency noise, low-frequency interference and baseline drift in the signal; and the downsampling method is used for the denoised signal to retain some sampling points in the original signal to reduce the sampling rate, and the downsampling rate is n.

5. The multimodal cardio-cerebral disease screening system based on manual feature guidance according to claim 1 is characterized in that: In the multi-granularity feature extraction module, coarse-grained feature extraction is performed on the preprocessed physiological signal data, specifically: EEG ECG signal Blood oxygen saturation blood pressure Respiratory rate Extract the general frequency domain feature F fre and the universal time domain feature F time , for ECG signals Extracted heart rate variability feature F hrv ; Common time domain features include maximum value, minimum value, mean value, standard deviation, variance, root mean square value, skewness, and kurtosis; Common frequency domain features include frequency mean, root mean square frequency, and standard deviation frequency; Heart rate variability features include RR interval standard deviation F extracted based on the time interval sequence between adjacent heart beats sdnn , RR interval root mean square F rmssd , the number of times the RR interval difference exceeds 50ms F nn50 And the percentage F pnn50 ; The general frequency domain feature F fre , general time domain feature F time 、Heart rate variability characteristics F hrv Splice to form coarse-grained manual features 6. The multimodal cardio-cerebral disease screening system based on manual feature guidance according to claim 1 is characterized in that: In the multi-granularity feature extraction module, the preprocessed physiological signal data is passed through a deep convolutional network to form a fine-grained signal feature matrix, specifically: Preprocessed multi-channel physiological signals A multi-layer deep convolutional network is used for layer-by-layer convolution. Each layer contains a pair of convolutional layers C1(·), C2(·) and a residual connection layer R(·). The following formula is used to calculate the multi-channel physiological signals. Perform fine-grained feature extraction to obtain fine-grained features F sig Where l represents the number of convolutional layers in the deep network.

7. The multimodal cardio-cerebral disease screening system based on manual feature guidance according to claim 1 is characterized in that: In the feature fusion module, the coarse-grained manual feature matrix is ​​weighted to the fine-grained feature matrix in the form of attention, specifically: The manual feature F hand Through the fully connected function f fc (·) Feature projection, the manual feature after projection is f fc (F hanc ) ; then calculate the weighted feature F guided by the coarse-grained manual features weight ; The weighted feature F wei and the projected manual feature f fc (F hand ) to obtain the fusion features 8. The multimodal cardio-cerebral disease screening system based on manual feature guidance according to claim 7 is characterized in that: The formula for calculating the weighted features guided by coarse-grained manual features is as follows: F weigh =softmax(f fc (F hand )·F sig T )·F sig In the formula, F sig It is a fine-grained feature.

Images