Ballistocardiogram ventricular fibrillation auxiliary diagnosis system based on three-channel images and transfer learning

Abstract

The present invention belongs to the field of medical instrument technology, specifically a ballistocardiogram ventricular fibrillation auxiliary diagnosis system based on three-channel images and transfer learning. The system of the present invention includes a signal preprocessing module, an image conversion module, a feature extraction module, a normalization processing module and a classifier; the present invention converts a one-dimensional BCG signal into three single-channel two-dimensional images in a customized manner, and then splices them into a three-channel image, which is used as an input tensor to migrate multiple CNN models pre-trained by a large image library to the BCG domain for learning, extract feature values ​​from the middle layer of each model and splice them into a one-dimensional feature vector, and then automatically distinguish ventricular fibrillation, sinus rhythm and motion artifacts through a fully connected layer or a machine learning classifier. The present invention can effectively solve the problem of difficulty in identifying ventricular fibrillation due to the diversity of BCG waveforms and the small sample size characteristics, and provide a feasible solution for long-term health monitoring of cardiovascular diseases at home.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical instruments, and in particular relates to a ballistocardiogram ventricular fibrillation auxiliary diagnosis system based on three-channel images and transfer learning. Background Art

[0002] Sudden cardiac death (SCD) is a major public health burden worldwide, and due to limited medical conditions, most SCD occurs outside of hospitals. Ventricular fibrillation is the primary cause of SCD, which is uncertain and highly critical. Once it occurs, ventricular output drops sharply and the patient's circulation is interrupted. If not intervened in time, it will cause sudden death within a few minutes. Defibrillation is the only effective method for treating ventricular fibrillation. Its success rate is inversely proportional to the duration of the ventricular fibrillation. For every minute of delay, the success rate of resuscitation drops by nearly 10%. Therefore, there is an urgent need for a non-contact real-time method to monitor cardiac function and accurately identify ventricular fibrillation.

[0003] At present, the clinical diagnosis of ventricular fibrillation is mainly based on electrocardiogram. However, electrocardiogram detection requires electrodes to be attached to the patient's body surface. Long-term use will cause inconvenience and discomfort to patients, so it is not suitable for long-term cardiac monitoring. The above problems can be solved by the non-interference and non-contact cardiac monitoring technology of ballistocardiogram (BCG). BCG uses sensors integrated in daily environments such as mattresses, cushions, and backrests to capture the movement of the body's center of mass during cardiac contraction and ejection, thereby providing users with overall performance information of the circulatory system. At present, the research on heart disease detection algorithms for BCG mostly uses traditional machine learning methods, which usually requires the extraction of various features of the signal. However, since BCG is easily affected by detection equipment, sitting and lying postures, and external vibration interference, its waveform diversity will increase and individual differences will be large, making the machine learning method based on feature engineering less universal. At the same time, since BCG currently lacks a relatively complete database, and deep learning models such as convolutional neural networks (CNN) have a large batch demand for input data sets in order to fully train and debug the network, it is not suitable for small sample studies such as BCG. In summary, a computer-aided diagnosis method for ventricular fibrillation suitable for BCG daily cardiac monitoring is needed to meet the needs of long-term unfettered health management for families with cardiovascular diseases. Summary of the invention

[0004] In view of the above-mentioned ventricular fibrillation detection algorithm requirements and the technical deficiencies of existing BCG research, the present invention provides a ballistocardiogram (BCG) ventricular fibrillation auxiliary diagnosis system based on three-channel images and transfer learning to solve the problem of difficulty in ventricular fibrillation identification caused by the diversity of BCG waveforms and small sample size characteristics.

[0005] The present invention converts one-dimensional BCG signals into three-channel images through a customized single-channel image combination method, and uses this as the input tensor. Multiple CNN models pre-trained by a large image library are migrated to the BCG domain for learning. The eigenvalues ​​are extracted from the middle layer of each model and spliced ​​into a one-dimensional feature vector, which is then used through a fully connected layer or a machine learning classifier to realize the discrimination of ventricular fibrillation, sinus rhythm and motion artifacts.

[0006] The BCG ventricular fibrillation auxiliary diagnosis system based on three-channel images and transfer learning provided by the present invention includes a signal preprocessing module, an image conversion module, a feature extraction module, a normalization processing module and a classifier; wherein:

[0007] The signal preprocessing module is used to preprocess the collected BCG signal, specifically including selecting the mother wavelet Daubechie 6 to decompose the original BCG signal, removing the high and low frequency noise components, and selecting the heartbeat related frequency band part (0.5-15Hz) for signal reconstruction.

[0008] The image conversion module is used to convert the one-dimensional BCG signal obtained by the signal preprocessing module into a three-channel two-dimensional image to construct a transfer learning input tensor. Specifically, three single-channel images are first obtained, including but not limited to a time-frequency diagram, a Gram angular field (GAF), and a custom autocorrelation sequence map (ACM). Among them:

[0009] The time-frequency image is obtained by obtaining the time-frequency coefficient matrix through short-time Fourier transform (STFT) or continuous wavelet transform (CWT), and then standardizing the matrix size through matrix compression transformation to obtain the required single-channel time-frequency grayscale image;

[0010] The GAF image is obtained by converting the one-dimensional BCG signal in the Cartesian coordinate system into a polar coordinate system, and then using trigonometric functions to generate a GAF matrix, thereby obtaining the required single-channel grayscale image;

[0011] The ACM image is obtained by reducing the one-dimensional BCG signal from a long sequence to a short sequence, calculating the autocorrelation coefficient sequence of the short sequence samples and converting it to an absolute value, normalizing the autocorrelation sequence value to a corresponding range and converting it to an integer with reference to the tensor size; then, the horizontal axis represents the time dimension, the vertical axis represents the amplitude dimension, and the processed autocorrelation sequence value is mapped to a two-dimensional grayscale image;

[0012] By concatenating the above three single-channel grayscale images along the channel dimension, we can obtain the three-dimensional input tensor required for transfer learning.

[0013] The feature extraction module uses the three-dimensional tensor obtained by the image conversion module as input and applies transfer learning to extract effective features; specifically, three classic CNN network models, VGG16, InceptionV3 and ResNet50, are pre-trained by the computer vision standard data set ImageNet, and the models are migrated to the three-dimensional input tensor obtained by the above-mentioned BCG conversion; feature maps are extracted from the middle layer of each model, and a one-dimensional feature vector is obtained after transformation and combination.

[0014] The normalization processing module is used to analyze and clean the feature vectors extracted by the feature extraction module through variance and mutual information parameters, and to normalize the features for use as subsequent classifier input.

[0015] The classifier adopts a fully connected layer, or a machine learning classifier such as logistic regression, support vector machine or random forest. The three types of signals, ventricular fibrillation, sinus rhythm and motion artifact, are divided into a training set and a test set by stratified sampling; the optimal hyperparameters of the model are obtained by cross-validation and grid search on the training set; on this basis, the model is trained again using the entire training set to determine the network structure and parameters of the classifier model; and the generalization performance of the final model is evaluated by the test set.

[0016] In the present invention, the construction of the three-channel image has the following specific process:

[0017] (1) The image size is specified as (N,N), where N is a custom value greater than 139;

[0018] (2) The preprocessed one-dimensional BCG signal is converted into an ACM image. The specific method is as follows:

[0019] ① According to formula (1), the long sequence data segment is reduced to a short sequence by the classic piecewise aggregation approximation (PAA) algorithm; the long sequence data segment is represented by x, with a length of L, and the short sequence is represented by x. paa Indicates that the length is N;

[0020] ②Calculate the short sequence sample x paa The autocorrelation coefficient sequence of is normalized to the range of [0, N-1] after absolute value conversion and integerization to obtain the sequence x acf ;

[0021] ③ Establish an all-zero matrix A of size (N,N) for each sample point x in the autocorrelation sequence acf [n], n = 0 ... N-1, update the matrix A according to formula (2), and use each data of the matrix A as the grayscale value of the corresponding pixel of the single-channel grayscale image ACM, including three levels of 0, 128, and 255;

[0022] in

[0023] Where I = x acf [n], n = 0...N-1, (2)

[0024] (3) Convert the preprocessed one-dimensional BCG signal into a GAF graph by:

[0025] ① The PAA algorithm is used to reduce the dimension of long sequence data segments into short sequences, with x paa Indicates that the length is N;

[0026] ②Normalize x paa , compress the sequence values ​​into the range [-1,1], with x norm express;

[0027] ③ According to the relationship shown in formula (3), the time series value x in the rectangular coordinate system is norm Convert to polar coordinate system (r, θ) and calculate the polar angle θ;

[0028] ④ m and n represent different time points, as shown in formula (4), and the trigonometric function transformation of angles and is applied to obtain the required GAF ​​matrix G. The matrix value is mapped to the range of [0, 255] to obtain a single-channel grayscale image;

[0029]

[0030] G[m][n]=cos(θ[m]+θ[n]), where m,n=0...N-1, (4)

[0031] (4) Convert the preprocessed one-dimensional BCG signal into a time-frequency image. The specific method is as follows: obtain the time-frequency coefficient matrix through STFT or CWT, adjust the matrix size to (N, N), and map the matrix value to the range of [0, 255] to obtain a single-channel grayscale image;

[0032] (5) The three single-channel images are combined into a (N,N,3) or (3,N,N) three-dimensional image in the form of one ACM image plus one GAF image plus one time-frequency image or other customized methods, and the grayscale value of each channel image is normalized to the range of [0,1] as the input tensor of the feature extraction module of subsequent transfer learning.

[0033] In the present invention, the feature extraction module applies transfer learning to extract effective features; the specific process is:

[0034] (1) Build three classic CNN network models: VGG16, InceptionV3, and ResNet50, and pre-train and tune the models using the computer vision standard dataset ImageNet;

[0035] (2) Freeze the above model parameters, use the three-dimensional tensor obtained after normalization of the custom three-channel image as the network input, and perform transfer learning; extract feature maps from the middle layer of each model, and obtain a one-dimensional feature vector after global average pooling and splicing.

[0036] In the present invention, the feature cleaning is carried out in a filtering way, firstly using 10 -8 As the threshold, remove low variance features; then remove 10% of low quality features based on the mutual information value. Each retained feature is normalized to the range of [0,1] as the input of the subsequent classifier.

[0037] In the present invention, the classifier uses the extracted features as input to distinguish ventricular fibrillation, sinus rhythm and motion artifacts. The classifier can be constructed by a fully connected layer or a machine learning classifier, wherein the fully connected layer uses a Softmax activation function as an output layer, uses the Adam algorithm for model optimization, and uses a cross entropy loss function to evaluate the quality; the machine learning classifier includes logistic regression, support vector machine or random forest.

[0038] The beneficial effect of the present invention is to provide a BCG ventricular fibrillation auxiliary diagnosis system based on three-channel images and transfer learning, which can use the BCG collected without sense to assist in identifying ventricular fibrillation attacks, and map the BCG from one dimension to a three-dimensional tensor by cleverly converting it. By using the richness of the open source image library and the maturity of the deep learning research related to computer vision, the classic model obtained by pre-training the ImageNet data set is migrated to the BCG measured data of small batch samples to distinguish between ventricular fibrillation and non-ventricular fibrillation. The algorithm provided by the present invention requires fewer retraining parameters, has strong model robustness, good diagnostic accuracy, and good practicality, providing a feasible idea for long-term health monitoring of cardiovascular diseases at home. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A system block diagram of an embodiment of the present invention.

[0040] Figure 2 The measured ventricular fibrillation BCG signal and the converted single-channel grayscale image are shown in the embodiment of the present invention. (a), (b), (c), and (d) are the one-dimensional ventricular fibrillation BCG signal and the ACM, GAF, and STFT grayscale images, respectively.

[0041] Figure 3The measured sinus rhythm BCG signal and the converted single-channel grayscale image of the embodiment of the present invention are shown in Figures (a), (b), (c), and (d) respectively show the one-dimensional sinus rhythm BCG signal and the ACM, GAF, and STFT grayscale images.

[0042] Figure 4 The measured motion artifact BCG signal and the converted single channel grayscale image of the embodiment of the present invention are shown in Figures (a), (b), (c), and (d) are the one-dimensional motion artifact BCG signal and the ACM, GAF, and STFT grayscale images, respectively.

[0043] Figure 5 This is a structural diagram of a feature extraction module for transfer learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following is a further description of specific embodiments of the present invention in conjunction with the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0045] In this embodiment, a BCG ventricular fibrillation auxiliary diagnosis system based on customized three-channel images and transfer learning is provided. Figure 1 As shown, the following steps are included:

[0046] Step 1: The signal preprocessing module preprocesses the signal to obtain a pure BCG signal.

[0047] Step 1.1: Manually extract motion artifact signals from BCG signals;

[0048] Step 1.2: Apply wavelet transform to filter out noise. Use the mother wavelet Daubechie 6 to decompose the original BCG signal into seven layers to obtain seven detail components (D1-D7). The detail components D3-D6 containing the heartbeat-related frequency band are recombined to obtain the filtered BCG signal;

[0049] Step 1.3: Divide the signal into 7-second segments.

[0050] Step 2: The image conversion module converts the image, and the image size is (256, 256).

[0051] Step 2.1: Convert the preprocessed one-dimensional BCG signal into an ACM image by:

[0052] ① According to formula (1), the PAA algorithm is used to reduce the dimension of the long sequence data segment (represented by x, with a length of L) into a short sequence, represented by x paa Indicates that the length is N = 256;

[0053] ②Calculate the short sequence sample x paaThe autocorrelation coefficient sequence of is normalized to the range of [0,255] after absolute value conversion and integerization to obtain the sequence x acf ;

[0054] ③ Establish an all-zero matrix A of size (256,256) for each sample point x in the autocorrelation sequence acf [n], n = 0...N-1, update the matrix A according to formula (2). The data of the matrix A are used as the grayscale value of the corresponding pixel of the single-channel grayscale image ACM, such as Figures 2 to 4 As shown in the neutron diagram (b);

[0055] in

[0056] Where I = x acf [n], n = 0…255, (2)

[0057] Step 2.2: Convert the preprocessed one-dimensional BCG signal into a GAF graph by:

[0058] ① The PAA algorithm is used to reduce the dimension of long sequence data segments into short sequences, with x paa Indicates that the length is 256;

[0059] ②Normalize x paa , compress the time series values ​​into the range [-1,1], with x norm express;

[0060] ③ According to the relationship shown in formula (3), the sequence value x in the rectangular coordinate system is norm Convert to polar coordinate system (r, θ) and calculate the polar angle θ;

[0061] ④ With m and n representing different time points, as shown in formula (4), the trigonometric function transformation of the angles and is applied to obtain the GAF matrix G required in this embodiment. The matrix values ​​are mapped to the range of [0, 255] to obtain a single-channel grayscale image, as shown in Figures 2 to 4 As shown in the neutron diagram (c);

[0062]

[0063] G[m][n]=cos(θ[m]+θ[n]), where m,n=0…255, (4)

[0064] Step 2.3: Convert the preprocessed one-dimensional BCG signal into a single-channel time-frequency (STFT) image. Use short-time Fourier transform, select the Hann window, set the window length to 2 seconds, and set the offset length to 0.04 seconds. Adjust the shape of the time-frequency coefficient matrix to (256, 256), and map the matrix values ​​to the range of [0, 255] to obtain the required single-channel grayscale image, such as Figures 2 to 4 As shown in the neutron diagram (d);

[0065] Step 2.4: Combine the three single-channel images into a (256, 256, 3) three-dimensional image in the manner of one ACM image, one GAF image, and one STFT image. Normalize the grayscale value of each channel image to the range of [0, 1] as the input tensor for subsequent transfer learning.

[0066] Step 3: The feature extraction module constructs a feature vector through transfer learning.

[0067] Step 3.1: Build three classic CNN network models: VGG16, InceptionV3 and ResNet50, and pre-train and tune the models using the computer vision standard dataset ImageNet; the feature extraction module structure of transfer learning is as follows Figure 5 As shown in the figure, FC, BN, ReLU, Conv2D, MaxPooling2D, GlobalAveragePooling2D, and ZeroPadding2D represent fully connected layers, batch normalization, ReLU activation layers, two-dimensional convolutional layers, two-dimensional maximum pooling layers, two-dimensional global average pooling layers, and two-dimensional zero padding layers, respectively; vgg_block, inception_block, and res_block represent the functional blocks of the three CNN models, and their internal structures are different. Since the above are all classic models, they are not explained in more detail here.

[0068] Step 3.2: Freeze the above model parameters, use the three-dimensional tensor obtained in step 2.4 as the network input, and perform transfer learning. Extract feature maps from the middle layer of each model, and obtain a one-dimensional feature vector with a length of 1056 after global average pooling and concatenation.

[0069] Step 4: The normalization processing module cleans and normalizes the features.

[0070] Use filtering method to clean features. First, -8 As the threshold, remove low variance features; then remove 10% of low quality features based on the mutual information value. Each retained feature is normalized to the range of [0,1];

[0071] Step 5: Classification and discrimination are performed by the classifier;

[0072] Step 5.1: Construct a classifier with a fully connected layer and use the Softmax activation function as the output layer; the model is optimized using the Adam algorithm and evaluated using the cross entropy loss function;

[0073] Step 5.2: The feature vector set obtained in step 4 constitutes the input of the classifier, and the output is the result of distinguishing whether the sample is ventricular fibrillation, sinus rhythm or motion artifact.

[0074] The above is a preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this example. Any other examples made by any technician familiar with the field within the technical scope disclosed by the present invention through simple replacement and substitution should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope defined in the claims.

Claims

1. A ballistocardiogram ventricular fibrillation auxiliary diagnosis system based on three-channel images and transfer learning, characterized in that: It includes a signal preprocessing module, an image conversion module, a feature extraction module, a normalization processing module and a classifier; wherein: The signal preprocessing module is used to preprocess the collected BCG signal; specifically, it includes selecting the mother wavelet Daubechie 6 to decompose the original BCG signal, removing the high and low frequency noise components, and selecting the 0.5-15 Hz frequency band part related to the heartbeat for signal reconstruction; The image conversion module is used to convert the one-dimensional BCG signal obtained by the signal preprocessing module into a three-channel two-dimensional image to construct a transfer learning input tensor; specifically, firstly obtain three single-channel images, including a time-frequency diagram, a Gram angle field GAF, and a custom autocorrelation sequence map ACM; wherein: The time-frequency diagram is obtained by: obtaining the time-frequency coefficient matrix through short-time Fourier transform STFT or continuous wavelet transform CWT; then standardizing the matrix size through matrix compression transformation, thus obtaining the required single-channel time-frequency grayscale diagram; The GAF image is obtained by converting the one-dimensional BCG signal in the Cartesian coordinate system into a polar coordinate system, and then using trigonometric functions to generate a GAF matrix, thereby obtaining the required single-channel grayscale image; The ACM image is obtained by reducing the one-dimensional BCG signal from a long sequence to a short sequence, calculating the autocorrelation coefficient sequence of the short sequence samples and converting it to an absolute value, normalizing the autocorrelation sequence value to a corresponding range and converting it to an integer with reference to the tensor size; then, the horizontal axis represents the time dimension, the vertical axis represents the amplitude dimension, and the processed autocorrelation sequence value is mapped to a two-dimensional grayscale image; Concatenate the above three single-channel grayscale images along the channel dimension to obtain the three-dimensional input tensor required for transfer learning; The feature extraction module uses the three-dimensional tensor obtained by the image conversion module as input and applies transfer learning to extract effective features; specifically, three classic CNN network models, VGG16, InceptionV3 and ResNet50, are pre-trained by the computer vision standard data set ImageNet, and the models are migrated to the three-dimensional input tensor obtained by the above BCG conversion; feature maps are extracted from the middle layer of each model, and a one-dimensional feature vector is obtained after transformation and combination; The normalization processing module is used to analyze and clean the feature vectors extracted by the feature extraction module through variance and mutual information parameters, and to normalize the features for use as input for subsequent classifiers; The classifier uses a fully connected layer, or uses logistic regression, support vector machine or random forest; divides the three types of signals, ventricular fibrillation, sinus rhythm and motion artifact, into a training set and a test set by stratified sampling; obtains the best hyperparameters of the model by cross-validation of the training set and grid search method; on this basis, uses the entire training set to train the model again, determines the network structure and parameters of the classifier model, and evaluates the generalization performance of the final model by the test set; The specific process of constructing the three-channel image is as follows: (1) The image size is specified as (N,N), where N is a custom value greater than 139; (2) The preprocessed one-dimensional BCG signal is converted into an ACM image. The specific method is as follows: ① According to formula (1), the long sequence data segment is reduced to a short sequence by the classic piecewise aggregation approximation (PAA) algorithm; the long sequence data segment is represented by x, with a length of L, and the short sequence is represented by x. paa Indicates that the length is N; ②Calculate the short sequence sample x paa The autocorrelation coefficient sequence of is normalized to the range of [0, N-1] after absolute value conversion and integerization to obtain the sequence x acf ; ③ Establish an all-zero matrix A of size (N,N) for each sample point x in the autocorrelation sequence acf [n], n = 0 ... N-1, update the matrix A according to formula (2), and use the data of the matrix A as the grayscale value of the corresponding pixel of the single-channel grayscale image ACM, including three levels of 0, 128, and 255; (3) Convert the preprocessed one-dimensional BCG signal into a GAF graph by: ① The PAA algorithm is used to reduce the dimension of long sequence data segments into short sequences, with x paa Indicates that the length is N; ②Normalize x paa , compress the sequence values ​​into the range [-1,1], with x norm express; ③ According to the relationship shown in formula (3), the time series value x in the rectangular coordinate system is norm Convert to polar coordinate system (r, θ) and calculate the polar angle θ; ④ m and n represent different time points, as shown in formula (4), and the trigonometric function transformation of angles and is applied to obtain the required GAF ​​matrix G. The matrix value is mapped to the range of [0, 255] to obtain a single-channel grayscale image; G[m][n]=cos(θ[m]+θ[n]), where m,n=0...N-1, (4) (4) Convert the preprocessed one-dimensional BCG signal into a time-frequency image. The specific method is as follows: obtain the time-frequency coefficient matrix through STFT or CWT, adjust the matrix size to (N, N), and map the matrix value to the range of [0, 255] to obtain a single-channel grayscale image; (5) The three single-channel images are combined into a (N, N, 3) or (3, N, N) three-dimensional image by means of a layer of ACM image plus a layer of GAF image plus a layer of time-frequency image or other customized methods, and the grayscale value of each channel image is normalized to the range of [0, 1] as the input tensor of the subsequent transfer learning feature extractor.

2. The ballistocardiogram ventricular fibrillation auxiliary diagnosis system according to claim 1, characterized in that: The feature extraction module applies transfer learning to extract effective features. The specific process is as follows: (1) Build three classic CNN network models: VGG16, InceptionV3, and ResNet50, and pre-train and tune the models using the computer vision standard dataset ImageNet; (2) Freeze the above model parameters and use the three-dimensional tensor obtained after normalization of the customized three-channel image as the network input for transfer learning; Feature maps are extracted from the middle layer of each model, and a one-dimensional feature vector is obtained through global average pooling and concatenation.

3. The ballistocardiogram ventricular fibrillation auxiliary diagnosis system according to claim 2, characterized in that: Feature cleaning, first with 10 -8 As the threshold, low variance features are removed; then, according to the mutual information value, 10% of low-quality features are removed; Each retained feature is normalized to the range of [0, 1] and used as the input of the subsequent classifier.

4. The ballistocardiogram ventricular fibrillation auxiliary diagnosis system according to claim 3, characterized in that: The classifier is constructed by a fully connected layer, in which the fully connected layer uses the Softmax activation function as the output layer, uses the Adam algorithm to optimize the model, and evaluates the quality with the cross entropy loss function.

Images