Human body sensing data processing method and system based on deep convolutional neural network

Abstract

The invention provides a human body sensing data processing method and system based on a deep convolutional neural network, and the method comprises the steps: obtaining electrocardiosignal data, dividing heart rhythm abnormity into two conditions: amplitude abnormity and interval abnormity, and processing the electrocardiosignal data into different color images according to different heart rhythm abnormity; inputting the color image into a VGG16 model for extracting features of different scales and abstract degrees in the image; and carrying out dimension reduction on the extracted features, and carrying out classification and visualization according to marks.

Description

Technical Field

[0001] This invention belongs to the field of human sensor data processing technology, and particularly relates to a method and system for human sensor data processing based on deep convolutional neural networks. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Cardiac arrhythmia refers to any irregular change in the normal heart rhythm, characterized by its sudden onset and intermittent nature. Severe arrhythmias can lead to cardiac arrest and death. Effective analysis and diagnosis of arrhythmias require the use of human sensory technologies such as electrocardiography (ECG) to detect heartbeat activity. ECG signals contain a wealth of information, providing rich insights for the diagnosis and analysis of diseases such as arrhythmias. However, traditional ECG analysis and diagnostic techniques rely heavily on manual labor; doctors must visually examine ECGs, which significantly reduces the timeliness and accuracy of the analysis. Therefore, developing a more effective and accurate arrhythmia analysis algorithm is a pressing issue that needs to be addressed.

[0004] There are three main types of electrocardiogram (ECG) signal analysis: (1) Combining machine learning methods, manual feature extraction can achieve relatively stable classification analysis. However, when the test set is uncertain or there is too much data, manual feature extraction is difficult to guarantee accuracy within a limited time.

[0005] (2) Template matching method is used. This method can effectively identify data with abnormal heart rhythm. However, this method is easily affected by noise in the data and may incorrectly identify noisy ECG signals as abnormal data, resulting in a large number of false positive results. This indicates that the reliability of this method is difficult to meet the requirements of clinical medicine.

[0006] (3) Analyze the changes in RR interval and waveform morphology. This method is easy to understand and implement, but the noise interference contained in the unprocessed data will cause a large error, and this method is difficult to solve dynamic electrocardiogram signal problems.

[0007] Therefore, existing ECG signal recognition methods have several limitations. First, traditional methods mainly rely on manual feature extraction, requiring expert domain knowledge and involving complex feature design, making it difficult to comprehensively capture the time-frequency characteristics of ECG signals. Second, traditional methods are sensitive to noise; baseline drift, power line interference, and electromyographic interference severely affect classification accuracy. Third, existing methods perform poorly when dealing with imbalanced data, exhibiting low accuracy in identifying rare arrhythmia types. Finally, traditional methods have high computational complexity and poor real-time performance, making them unsuitable for the needs of rapid clinical diagnosis.

[0008] In recent years, with the continuous advancement of big data science and deep learning technology, deep learning methods have performed exceptionally well in image classification and recognition, automatically extracting features, avoiding manual intervention, and improving analysis efficiency and accuracy.

[0009] Electrocardiogram (ECG) signal classification is an important research area in life sciences, and deep learning methods have demonstrated powerful capabilities in this field. However, training a classification model applicable to medical practice requires a large amount of clinical data. This is because there are several problems and difficulties in model training. The most critical issue is the imbalance and sparsity of the dataset. Typically, there is a large amount of data on normal conditions, while data on arrhythmias is scarce, making it difficult for neural network models to extract sufficient features from the data. Moreover, the cost of misclassifying arrhythmias is higher than that of misclassifying normal conditions. This necessitates that the model avoid false positives as much as possible. On the other hand, the quantity and cost of clinical datasets are also issues. Medical datasets such as ECGs are often limited in number and expensive. This restricts the scale and economic benefits of dataset acquisition, which in turn limits the application scope and widespread adoption of deep learning methods in ECG signal classification.

[0010] Deep learning algorithms have achieved remarkable results in research areas such as image classification, recognition, and prediction. However, human sensor data is usually one-dimensional, and without using complex models, the classification results are neither ideal nor intuitive. Summary of the Invention

[0011] To overcome the shortcomings of the prior art, this invention provides a human sensor data processing method based on deep convolutional neural networks. Using deep convolutional neural networks can save unnecessary time and manpower costs, while preserving the information contained in the data to the greatest extent, thereby achieving stable, accurate and real-time classification and analysis of electrocardiogram signals.

[0012] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: Firstly, a method for processing human sensor data based on deep convolutional neural networks is disclosed, including: Acquire electrocardiogram (ECG) signal data and classify arrhythmias into two types: amplitude abnormalities and interval abnormalities. Process the ECG signal data into different color images for different arrhythmias. The color image is input into the VGG16 model to extract features of different scales and levels of abstraction in the image. The extracted features are reduced in dimensionality and then classified and visualized based on the labels.

[0013] As a further technical solution, for interval abnormalities, the electrocardiogram signal is converted into a color two-dimensional image. The two-dimensional image contains the signal at a set time, is divided into multiple rows, and each row is set with a number of points. The color represents the amplitude, which is used to preserve the time and amplitude information of the electrocardiogram signal. The two-dimensional images are divided into two categories: normal and abnormal, and the training set and test set are divided according to a set ratio.

[0014] As a further technical solution, for amplitude abnormalities, the electrocardiogram signal is converted into a time-frequency image. Each time-frequency image contains a signal of one cycle, with the horizontal axis representing time, the vertical axis representing frequency, and the color representing amplitude. The frequency domain features of the electrocardiogram signal are extracted based on the time-frequency image to reflect changes in cardiac electrical activity. The time-frequency images are divided into two categories: normal and abnormal. The training set and test set are divided according to a certain ratio. The time-frequency images are then input into the neural network for training and testing.

[0015] As a further technical solution, the VGG16 model is a convolutional neural network for transfer learning, which is a deep learning model trained on a large-scale dataset. The convolutional neural network consists of two parts: a feature extraction part and a classifier part; The feature extraction part uses the convolutional and pooling layers in the VGG16 model as a convolutional neural network for transfer learning.

[0016] As a further technical solution, image features are extracted through convolutional and pooling layers in the VGG16 model. Then, the feature vectors are flattened into one-dimensional vectors through a Flatten layer. The feature vectors are then mapped through two fully connected layers. The fully connected layers use the ReLU activation function to add non-linearity. After another fully connected layer, the feature vectors are mapped again. Finally, the softmax activation function is input to normalize the vectors into a probability distribution. The binary cross-entropy loss function is calculated to measure the difference between the predicted results and the true labels.

[0017] Secondly, a human sensor data processing system based on deep convolutional neural networks is disclosed, including: The electrocardiogram signal data processing module is configured to: acquire electrocardiogram signal data and classify arrhythmias into two types: amplitude abnormalities and interval abnormalities; and process the electrocardiogram signal data into different color images for different arrhythmias. The feature extraction module is configured to input a color image into the VGG16 model to extract features of different scales and levels of abstraction in the image. The classification module is configured to reduce the dimensionality of the extracted features and classify and visualize them based on the labels.

[0018] The above one or more technical solutions have the following beneficial effects: This intelligent ECG signal analysis method, based on deep convolutional neural networks and transfer learning, transforms one-dimensional ECG signals into two-dimensional images. By combining transfer learning techniques, it effectively addresses the problems of data sparsity and difficulty in feature extraction inherent in traditional ECG signal analysis methods. This method can more intuitively display the characteristics of ECG signals and fully utilize the knowledge of pre-trained models, thereby improving classification accuracy and efficiency, achieving a technological breakthrough in ECG signal analysis. Using deep convolutional neural networks can save unnecessary time and manpower costs while preserving the information contained in the data to the greatest extent, thus enabling stable, accurate, and real-time classification analysis of ECG signals.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0021] Figure 1 This is a two-dimensional image diagram illustrating abnormal electrocardiogram signal intervals according to an embodiment of the present invention; Figure 2 This is a two-dimensional image diagram illustrating abnormal electrocardiogram signal amplitude according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the softmax layer structure in an embodiment of the present invention; Figure 4 A set of images of normal electrocardiogram signals; Figure 5 Data marked as normal signals but whose waveforms cannot be identified; Figure 6 Image set containing abnormal heart rate intervals; Figure 7 ECG signal classification results; Figure 8 Accuracy line graph; Figure 9 Loss value line chart; Figure 10 An image set containing abnormal heart rate amplitudes; Figure 11 A set of images of normal electrocardiogram signals; Figure 12 ECG signal classification results; Figure 13 Abnormal electrocardiogram (ECG) signal images; Figure 14Abnormal electrocardiogram waveform; Figure 15 A classification chart showing the visualization results after changing the dataset; Figure 16 Accuracy variation graph; Figure 17 A graph showing the change in the loss value. Detailed Implementation

[0022] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0024] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0025] The main process for analyzing electrocardiogram (ECG) signals is as follows.

[0026] (1) ECG signal preprocessing: ECG signals are weak bioelectrical signals. ECG data collected in clinical recordings often contain contamination from various sources of noise, such as 50 Hz / 60 Hz power frequency interference caused by the magnetic field of the power supply acting on the loop circuit between the ECG machine and the human body, electromyographic interference caused by the patient's muscle tension, frequency response caused by the patient's respiratory movements or by the interface impedance between electrodes, and baseline drift, generally less than 1 Hz. These noises can distort the ECG signal waveform, reduce signal quality, and cause difficulties for subsequent signal processing and diagnosis. Therefore, noise reduction processing of ECG signals is a necessary and important task.

[0027] Currently, there are various methods for denoising ECG signals. Digital filter-based methods primarily utilize the frequency domain characteristics of the ECG signal to design appropriate cutoff frequencies or adaptive parameters, thereby filtering out noise components. Common filters include morphological filters, finite impulse response filters, median filters, and adaptive filters. These methods are simple to implement, but they also have some drawbacks, such as potential loss of signal integrity and inability to effectively handle nonlinear or non-Gaussian noise.

[0028] Wavelet transform-based methods primarily utilize wavelet functions to perform multi-scale decomposition and reconstruction of electrocardiogram (ECG) signals, thereby suppressing or eliminating noise components. Common wavelet denoising methods include wavelet thresholding and Empirical Mode Decomposition (EMD). These methods can preserve the local characteristics of the signal in the time-frequency domain and possess good adaptability and flexibility. However, they also have some drawbacks, such as the selection of thresholds and wavelet basis functions, and high computational complexity.

[0029] In addition to the two types of methods mentioned above, there are other types of ECG signal denoising methods, such as collaborative filtering methods based on similar segments and deep learning methods based on recurrent neural networks.

[0030] (2) ECG signal feature extraction: The features in ECG signals can reflect the physiological and pathological state of the heart. Therefore, feature extraction is an important step in ECG signal classification and recognition. The more accurate the extracted features are, the higher the accuracy of the final classification and recognition.

[0031] Generally, electrocardiogram (ECG) signals mainly consist of the P wave, QRS complex, and T wave, with other features derived from these. Among these, the QRS complex is the most prominent and stable part of the ECG signal, reflecting the ventricular depolarization process. Detecting the QRS complex is relatively easy; commonly used methods include the wavelet transform modulus extremum method and the energy level and rate of change threshold method. The wavelet transform modulus extremum method uses the combination of extrema and zero-crossing points after wavelet transform of the original ECG signal to detect the QRS complex. The energy level and rate of change threshold method detects the QRS complex by setting thresholds for the amplitude and duration of the energy level and rate of change.

[0032] The P and T waves are weaker and less stable components of the electrocardiogram (ECG) signal, reflecting atrial depolarization and ventricular repolarization. Detecting the P and T waves is relatively difficult, requiring consideration of factors such as noise, baseline drift, and morphological variations. In recent years, some researchers have proposed novel methods to extract the P and T waves. For example, a method combining Markov algorithms and Bayesian methods is used. This method segments the ECG signal using a Markov algorithm, then uses a Bayesian method to estimate the probability of P and T waves in each segment, and finally determines the location of the P and T waves based on the probability maximization principle. Additionally, Kresimir et al. proposed a method based on the peak characteristics of the R wave and the P and T waves for detection. This method first locates the R wave peak in the ECG, then determines the RR interval, and finally retrieves the P and T waves within that range based on their peak characteristics. Furthermore, Panigrahy D proposed a method based on a differential evolutionary optimization strategy for P wave detection. This method utilizes a differential evolution algorithm to optimize a phase assignment function, and then determines the location of the P-wave based on the extreme points of this function. This method shows good accuracy in P-wave detection; however, it is computationally intensive, and the phase assignment method described in this paper relies on the accurate detection of the R-peak.

[0033] The characteristics of different cardiac arrhythmias are complex and varied. Extracting only a single feature or simply fusing multiple features is insufficient to distinguish arrhythmias; it is necessary to construct and select signal features based on prior knowledge. For example, analyzing the width of the QRS complex can determine whether there are premature ventricular contractions (PVCs), and analyzing the P wave can determine whether there is atrial fibrillation. Mahmoodabadi et al. used a multi-resolution wavelet transform method to extract ECG signal features. This method decomposes the ECG signal into sub-band signals of different frequencies, then extracts statistical features such as energy, entropy, and standard deviation from each sub-band signal, and finally combines these features into a feature vector.

[0034] (3) ECG signal classification algorithm: ECG signal analysis requires extensive experience and consumes a lot of time. In order to assist doctors in the diagnosis and treatment of arrhythmias and improve their efficiency and accuracy, the use of computer technology to automatically identify and judge ECG signals has become a hot research direction.

[0035] Traditional machine learning algorithms require preprocessing, feature extraction, and feature selection of ECG signals before classifying them using classifiers such as Support Vector Machines (SVMs), Decision Trees, Random Forests, and K-Nearest Neighbors. The advantages of algorithms like SVMs, Artificial Neural Networks (ANNs), and Random Forests (RF) are their maturity and stability; however, they require manual feature design and are significantly affected by noise and interference.

[0036] Deep learning, as an end-to-end solution, can automatically extract features from raw signals, avoiding human intervention and errors. Deep learning methods have shown superior performance in ECG signal classification, but several challenges and problems remain. The most significant are data imbalance and insufficient accuracy in classifying sparse arrhythmias. Because some life-threatening arrhythmias are very rare, the number of normal and arrhythmic heartbeats in the training data varies greatly, leading to weaker recognition capabilities of deep learning models for a few categories.

[0037] To address these issues, some researchers have proposed different deep learning models and data transformation methods, and conducted experiments and evaluations on various databases. For example: Wang et al. proposed an end-to-end deep learning method for multi-class arrhythmia detection with multi-stage feature fusion. They used the MIT-BIH arrhythmia database and the CinC2017 database for training and testing, achieving an accuracy of 97.5% and an F1 score of 95.9% across 12 arrhythmia detection classes.

[0038] Lin et al. proposed an automated system combining convolutional neural networks and long short-term memory networks. The novelty of this work lies in its use of variable-length ECG segments from the MIT-BIT arrhythmia database. They achieved an accuracy of 99.3% and an F1 score of 98.8% in the detection of five types of arrhythmias.

[0039] Rajpurkar P et al. proposed a 34-layer ResNet deep learning model, which they used to map electrocardiogram (ECG) sample sequences to rhythm sequences without any complex preprocessing or feature engineering steps. They claim that the model's ECG discrimination ability surpasses that of ordinary cardiologists. They used a large database containing 64,121 ECG records, encompassing 14 types of arrhythmias.

[0040] Gao et al. proposed using a long short-term memory network model with focus loss to classify imbalanced arrhythmia data. They used the MIT-BIH arrhythmia database and the MIT-BIH echocardiography database for training and testing, achieving an accuracy of 98.6% and an F1 score of 96.8% in the detection of four types of arrhythmias.

[0041] Essa E et al. applied ensemble learning to ECG signal classification, proposing a deep learning-based multi-model system that combines the classification results of these models into a final prediction. They used the MIT-BIH arrhythmia database and the PTB diagnostic ECG database for training and testing, achieving an accuracy of 99.2% and an F1 score of 98.9% in five arrhythmia detection categories.

[0042] Some researchers use short-term Fourier transform and wavelet transform to convert ECG data into two-dimensional (frequency, time) data and use it as input to deep neural networks. This method can leverage the characteristics of the signal in the frequency domain to improve classification performance. For example: Salem et al. used the MIT-BIH dataset to convert one-dimensional ECG signals into “spectral maps” to create 2D images. They also used a 161-layer DenseNet, pre-trained on millions of images to extract abstract information, and then applied SVM for four-class classification. Their model achieved an accuracy of 97.23% and an F1 score of 97.35%. However, this method is computationally complex and requires a large amount of disk space to store the 2D images.

[0043] Despite some progress in electrocardiogram (ECG) signal classification, the following technical challenges remain: (1) Most methods treat ECG signals as one-dimensional sequences, failing to fully utilize the time-frequency characteristics of the signals.

[0044] (2) Many methods require a large amount of labeled data for training, while medical data is expensive to acquire and difficult to label.

[0045] (3) Existing methods have limited effectiveness in dealing with data imbalance problems and are insufficient in identifying rare arrhythmia types.

[0046] (4) Most methods lack effective noise processing mechanisms and have poor robustness in actual clinical applications.

[0047] This embodiment applies deep learning algorithms to image classification and recognition of human sensor data. Specifically, it involves converting one-dimensional human sensor data into two-dimensional or three-dimensional images, and then using a convolutional neural network model for classification and recognition. By combining image classification and recognition with deep learning methods, more layers of information can be obtained intuitively and accurately. Experimental data is preprocessed using methods such as denoising before being input into a deep neural network model to obtain the feature matrix of electrocardiogram signals. This processing method can directly represent the characteristics present in human sensor data. Finally, this embodiment verifies that deep learning algorithms can be effectively applied to image classification and recognition of human sensor data, while further analysis and mining of human sensor data are achieved based on deep convolutional neural networks.

[0048] Example 1 This embodiment discloses a human sensor data processing method based on deep convolutional neural networks. The method sequentially performs wavelet denoising, two-dimensional image generation, inputting a pre-trained VGG16 model (transfer learning) for feature extraction, extracting high-dimensional features, PCA dimensionality reduction, inputting a customized classifier, and outputting the classification result. Specific steps include: Step 1: Process human sensor data into color images. Since human sensor data is usually a one-dimensional or two-dimensional numerical sequence, while deep learning neural networks require three-dimensional image input, the data needs to be converted into a suitable format. This example uses a method to map the data to a color space, so that each data point corresponds to a color pixel. Here, human sensor data refers to one-dimensional electrocardiogram signals.

[0049] Standardization: Standardize the signal, such as making the signal amplitude between -1 and 1; Input to VGG16 model: Input the preprocessed two-dimensional image into the VGG16 model for feature extraction.

[0050] Step 2: Use the VGG16 model as the convolutional neural network for transfer learning to extract image features. Transfer learning is a method that uses a deep learning model trained on a large-scale dataset to fine-tune or adapt it for a new task. The VGG16 model, through multiple convolution and pooling operations, can effectively extract features of different scales and levels of abstraction from images and has strong generalization ability. The VGG16 model employs transfer learning, utilizing its pre-trained weights.

[0051] Step 3: Use the PCA algorithm to reduce the dimensionality of the extracted features, and then classify and visualize them based on the labels. Extract the second fully connected layer from the VGG16 model, expand the feature matrix into a 4096-dimensional vector, use PCA to reduce the dimensionality of the feature vector, and then select an appropriate number of principal components to obtain a low-dimensional feature vector. Use the dimensionality-reduced feature vector for subsequent classification tasks. This method leverages the powerful feature extraction capabilities of the VGG16 model while reducing feature redundancy and noise, improving the model's efficiency and accuracy. Different categories of data exhibit clear clustering effects on the two-dimensional plane.

[0052] Features are extracted from specific layers of the VGG16 model (such as the fully connected second layer, which outputs a 4096-dimensional vector), and the extracted feature vectors are reduced in dimensionality (e.g., to 5 dimensions) using the PCA algorithm.

[0053] Classification: The dimensionality-reduced features are input into a custom classifier (fully connected layer + softmax layer) for classification. The model is trained using the binary cross-entropy loss function and the Adam optimizer.

[0054] In this implementation example, one-dimensional electrocardiogram (ECG) signals (especially those employing different imaging strategies for different abnormality types, such as time-domain structured images of interval abnormalities and time-frequency images of amplitude abnormalities) are transformed into two-dimensional images. Then, a mature image recognition CNN (VGG16) is applied for analysis. This cross-modal processing approach is itself an innovative application in the field of ECG signal analysis. It solves the problems of traditional one-dimensional signal processing methods' difficulty in intuitively capturing complex patterns and the poor performance of deep learning models directly processing short one-dimensional sequences.

[0055] Applying transfer learning to ECG signal analysis, which involves a complete process of specific denoising (wavelet thresholding), specific visualization (STFT spectrograms or structured time-domain plots), and specific feature dimensionality reduction (PCA processing of VGG16 output), is an effective strategy to address the problems of data sparsity and high training costs in this field.

[0056] This application presents a complete technical workflow from signal preprocessing (wavelet denoising), data format transformation (2D imaging), feature extraction (transfer VGG16), feature refinement (PCA dimensionality reduction) to classification. Each step in this workflow works in concert to improve the accuracy of ECG signal classification (95.36% and 93.11%) and reduce reliance on large-scale data. This systematic integration and optimization, especially the customized steps for ECG data characteristics (such as targeted imaging methods), surpasses the conventional application of single technologies such as VGG16 or PCA.

[0057] This method effectively improves the accuracy of ECG signal classification. At the same time, through transfer learning and image processing, it reduces the need for large-scale raw ECG data, alleviates the problem of insufficient data, and greatly reduces the scale of ECG signal data required to train the model. This effectively improves the accuracy of ECG signal classification and reduces the errors caused by insufficient data in the ECG signal database.

[0058] In this implementation example, a significant technological breakthrough was achieved in the ECG signal classification method based on two-dimensional image visualization and transfer learning. By converting one-dimensional ECG signals into two-dimensional images, the time-frequency characteristics of the signals are preserved, enabling CNNs to extract key information more effectively. The transfer learning of the VGG16 model addresses the problem of scarce medical data, while preprocessing methods such as wavelet thresholding denoising and PCA dimensionality reduction further improve the model's robustness and classification accuracy.

[0059] To effectively classify and identify human sensor data, this embodiment uses the VGG16 model as a convolutional neural network for transfer learning. It leverages pre-trained VGG16 model parameters on the ImageNet dataset to accelerate the training process and improve classification performance. For data preprocessing, wavelet thresholding is used to remove noise from the electrocardiogram (ECG) signals, and PCA is used to reduce the dimensionality of extracted features, thereby reducing data redundancy and noise and improving model efficiency and accuracy.

[0060] In one implementation example, in step one, raw data acquisition: ECG signal data are selected from the MIT-BIH arrhythmia database, and noise preprocessing is performed: the raw one-dimensional ECG signal is processed using wavelet thresholding to eliminate noise such as power line interference, electromyography interference, and baseline drift. Before processing the one-dimensional electrocardiogram (ECG) signal into a color image, arrhythmias were classified into two types: amplitude abnormalities and interval abnormalities, and classification and recognition experiments were conducted separately. Amplitude abnormalities refer to ECG signal peaks or troughs exceeding the normal range, while interval abnormalities refer to time intervals between different waveforms or between each cycle of the ECG signal that are too short or too long.

[0061] For cases of abnormal intervals, the ECG signal is converted into a color two-dimensional image; see appendix. Figure 1As shown, each image contains 10 seconds of signal, divided into 5 rows with 600 points per row, and color represents amplitude. This method preserves the temporal and amplitude information of the ECG signal while utilizing image processing techniques for classification. The image data is divided into two categories: normal and abnormal, with the training and test sets split in a 7:3 ratio, totaling 88 images. The image data is then input into a transfer learning model for training and testing. A transfer learning model refers to using a pre-trained deep learning model and fine-tuning or adjusting it based on a new dataset. This method leverages existing model knowledge to improve classification accuracy and efficiency.

[0062] For cases of abnormal amplitude, the ECG signal is converted into a time-frequency image. Each image contains one cycle of the signal, with the horizontal axis representing time, the vertical axis representing frequency, and color representing amplitude. This method can extract the frequency domain features of the ECG signal, reflecting changes in cardiac electrical activity. The time-frequency images are divided into two categories: normal and abnormal, and the training and test sets are divided according to a certain ratio. The time-frequency images are then input into a neural network for training and testing. A neural network is a computational model that mimics the structure and function of the human brain and can achieve complex nonlinear mappings by learning from large amounts of data. This method can utilize the powerful fitting ability of neural networks to achieve efficient and flexible classification. The converted ECG signal image is shown below. Figure 2 As shown.

[0063] To classify electrocardiogram (ECG) signals, a convolutional neural network model was designed for this example; see appendix. Figure 3 As shown, the model consists of two parts: a feature extraction part and a classifier part. The feature extraction part uses the convolutional and pooling layers from the original VGG16 model as a convolutional neural network for transfer learning. The classifier part consists of three fully connected layers and one softmax layer for classification, with output dimensions of 1024, 256, and 64 dimensions respectively. The ReLU function has advantages such as nonlinearity, sparsity, and gradient non-saturation, which can improve the model's expressive power and generalization ability.

[0064] This example uses the binary cross-entropy loss function as the model's loss function. The binary cross-entropy loss function is a commonly used loss function for evaluating binary classification problems; it measures the difference between the predicted probability and the true label. The loss value increases significantly when the prediction is incorrect and the confidence level is high. The Adam optimizer is used to optimize the model's parameters. The Adam optimizer is an adaptive gradient descent method that dynamically adjusts the learning rate and the decay rates of the first and second moments, thereby improving training efficiency and stability. This paper trains the model for 30 cycles and visualizes the accuracy and loss for each cycle to observe the training process and effect. Experimental results show that this method can effectively identify ECG signals with abnormal heart rate intervals and abnormal heart rate amplitudes, achieving high accuracy and stability.

[0065] In this implementation example, three fully connected layers and one softmax layer can be added to the VGG16 layer to output the classification results. The data passes through the VGG16 convolutional and pooling layers to extract image features, then through a Flatten layer to flatten the feature vectors into one-dimensional vectors. These feature vectors are then passed through two fully connected layers, mapping them to vectors of size 1024 and 128 respectively. The fully connected layers use the ReLU activation function to add non-linearity, as shown in the formula: (1) The feature vector is then passed through a fully connected layer, mapping it to a vector with 64 classes. Next, a softmax activation function normalizes the vector to a probability distribution, and a binary cross-entropy loss function is calculated to measure the difference between the predicted result and the true label. The classifier uses the Adam optimizer with parameters set to... , , , Update the network parameters and minimize the binary cross-entropy loss function. Set the number of iterations n to 30.

[0066] Experimental setup and analysis, evaluation metrics: In the process of ECG signal classification and recognition, normal data account for a relatively high proportion. Due to the uneven distribution of samples, four metrics were set up in the experiment to evaluate the performance of the model: Accuracy (ACC), Precision (Pre), Recall (Rec), and F1 score (F1). The calculation formulas for each metric are as follows: (2) (3) (4) (5) in TP A true positive indicates the frequency with which a positive sample was successfully predicted as positive: FN A false negative indicates the frequency with which a positive sample is incorrectly predicted as negative. FP A false positive indicates the frequency with which a negative sample is incorrectly predicted as positive. TN A true negative indicates the frequency with which a negative sample was successfully predicted as negative.

[0067] Experimental Procedure: Electrocardiogram (ECG) signal data were selected from the MIT-BIH Heart Rate Abnormality Database, and the data underwent noise reduction and standardization. The ECG signal data were then converted into two-dimensional images. These images were input into the VGG16 model, and using transfer learning and feature dimensionality reduction techniques, accurate classification was achieved between ECG signal images with abnormal ECG intervals and those with abnormal ECG amplitudes and normal ECG signal images.

[0068] Classification experiment of abnormal heart rate intervals: To identify electrocardiogram signals with abnormal heart rate intervals, a method based on image processing and the VGG16 model was used. The specific steps are as follows: First, two types of electrocardiogram (ECG) signal data were selected from the MIT-BIH Heart Rate Abnormality Database: one type of normal signal and the other type of signal with abnormal heart rate intervals. Abnormal heart rate intervals refer to irregular time intervals between heartbeats, which may lead to arrhythmias or other heart problems.

[0069] Secondly, the ECG signal data underwent noise reduction and standardization to remove high-frequency noise and baseline drift, ensuring the signal amplitude was between -1 and 1. Then, the time-domain characteristics of the signal were converted to frequency-domain characteristics using Short-Time Fourier Transform (STFT), yielding the signal's spectrogram, which served as the input image. The spectrogram reflects the signal's frequency distribution and energy density, aiding in the extraction of signal features.

[0070] The input image is then fed into the VGG16 model for feature extraction and classification. The VGG16 model is a classic convolutional neural network model consisting of 16 layers, including 13 convolutional layers and 3 fully connected layers. Transfer learning is used, leveraging pre-trained VGG16 model parameters on the ImageNet dataset to accelerate the training process and improve classification performance. A softmax classifier is then added to the last layer of the VGG16 model to output probability values ​​for two categories (normal or abnormal), and the category is determined based on the highest probability.

[0071] Next, the output of the second fully connected layer was extracted from the VGG16 model as a feature vector, with a dimension of 4096. Principal Component Analysis (PCA) was used to reduce the dimension of the feature vector to 5, preserving the maximum amount of information. Visualization images were generated based on the first two principal components, observing the distribution of different categories of signal images on a two-dimensional plane. The results showed that normal and abnormal signal images could be clearly distinguished.

[0072] Finally, the model's classification performance was evaluated using metrics such as accuracy, precision, recall, and F1 score. The dataset was divided into training and testing sets for training and testing the model, respectively. Four-fold cross-validation was also used to enhance the model's generalization ability and persuasiveness. Finally, the accuracy and loss values ​​during training were plotted as curves to observe the model's convergence.

[0073] Experimental results show that the proposed method can effectively identify ECG signals with abnormal heart rate intervals and achieves high classification performance. Specific results are as follows: Figure 4 This is a normal two-dimensional image dataset. As you can see, the effect of each image is basically the same, showing clear spectral characteristics.

[0074] Figure 5 These images, marked as normal but with indistinguishable waveforms, may be due to instability or interference during ECG signal acquisition. Such images may cause misclassification or reduce accuracy during classification, thus requiring further processing or exclusion. Figure 6 This is a dataset of two-dimensional images showing abnormal heart rate intervals. These images differ significantly from normal images, primarily in spectral distribution and energy density. They can be effectively distinguished during classification.

[0075] Figure 7 This is a visualization of the classification results. Red dots represent normal signal images, and blue dots represent abnormal signal images. As you can see, there is a clear boundary between the two types of images on the two-dimensional plane, indicating that the model can effectively extract and distinguish features.

[0076] Figure 8 and Figure 9The graph shows the accuracy and loss values, indicating that the model gradually converged during training, achieving high accuracy and low loss. The model achieved an accuracy of 96.77% on the training set, 94.97% on the test set, a precision of 95.36%, a recall of 94.97%, and an F1 score of 95.59%. After using four-fold cross-validation, the mean squared error (MSE) was [0.022, 0.008, 0.011, 0.043], and the mean absolute error (MAE) was [0.110, 0.047, 0.081, 0.066]. The error of the classification algorithm approaches zero, indicating that the algorithm performs well in classifying abnormal heart rate intervals.

[0077] To identify electrocardiogram (ECG) signals with abnormal heart rate amplitude, a method based on Fourier transform and the VGG16 model was employed. The specific steps are as follows: First, a Fourier transform was performed on the electrocardiogram (ECG) signal data to convert the signal's time-domain characteristics into frequency-domain characteristics, resulting in a spectrum, which served as the input image. The spectrum reflects the signal's frequency distribution and energy density, aiding in the extraction of signal features.

[0078] Next, the input images were fed into the VGG16 model for feature extraction and classification. The distribution of different categories of signal images on a two-dimensional plane was observed. The experiments showed that normal and abnormal signal images could be clearly distinguished.

[0079] Experimental results show that the method can effectively identify ECG signals with abnormal heart rate amplitude and achieves high classification performance. Specific results are as follows: Figure 10 This is a spectrogram dataset with abnormal heart rate amplitudes, showing significant differences from normal images, primarily in frequency distribution and energy density. These images can be effectively distinguished during classification. Figure 11 This is a normal spectrogram dataset. As you can see, the effect of each image is basically the same, presenting clear frequency domain characteristics.

[0080] Figure 12 This is a visualization of the preliminary classification results. Red dots represent normal signal images, and blue dots represent abnormal signal images. As you can see, there are multiple red dots within the blue area, indicating that there may be abnormal samples in the data, affecting the accuracy of classification and recognition. Figure 13 and Figure 14These are signal features marked as normal but whose waveforms cannot be identified, possibly due to instability or interference during ECG signal acquisition. These images may cause misclassification or reduce accuracy during classification, thus requiring further processing or exclusion. Therefore, this experiment removed these data, re-extracted features, and then reduced the dimensionality.

[0081] Figure 15 It is a visualization of the classification results. Figure 16 and Figure 17 The graph shows the accuracy and loss values. It can be seen that after removing abnormal samples, the model gradually converged during training, achieving high accuracy and low loss. The model's accuracy on the training set was 94.52%, and its accuracy, precision, and recall on the test set were 93.11%, 93.23%, and 93.14%, with an F1 score of 92.79%. After using four-fold cross-validation, the mean squared errors were [0.019, 0.068, 0.191, 0.051], and the mean absolute errors were [0.071, 0.139, 0.302, 0.121]. It can be observed that the classification algorithm's error approaches 0, indicating that the algorithm performs accurately in classifying abnormal heart rate amplitudes.

[0082] This experiment applies a deep learning neural network model to physiological signals, specifically electrocardiogram (ECG) signals. Considering that ECG signals can be processed in the frequency domain, and combining this with time-domain variations, an image-based processing of the ECG signal can be achieved. This study primarily focuses on two types of cardiac arrhythmias, employing different image-based methods and neural network models for classification and identification. For interval abnormalities, the time-domain ECG signal is converted into a color two-dimensional image, and a transfer learning model is used for training and testing. For amplitude abnormalities, the ECG signal undergoes a Fourier transform to obtain a time-frequency image, which is then used for neural network training and testing. Experimental results show that image-based processing can effectively extract the time-frequency information of ECG signals, thereby achieving the classification and identification of cardiac arrhythmias.

[0083] The challenges and solutions presented in this paper regarding data volume, data quality, feature extraction, and model training are as follows: Data Volume: The amount of ECG signal data is often limited by factors such as acquisition equipment, acquisition time, and the subjects being acquired, resulting in insufficient available data to support the training and testing of deep learning models. Our proposed method utilizes transfer learning techniques, borrowing a pre-trained VGG16 model to reduce reliance on large amounts of data and improve the model's generalization ability and training efficiency. Furthermore, our method can also expand the dataset through data augmentation techniques such as rotation, scaling, cropping, and noise reduction, increasing data diversity and robustness.

[0084] Data Quality: The quality of ECG signal data is often affected by noise, interference, and anomalies, leading to signal distortion or incompleteness, which impacts feature extraction and classification. Our proposed method improves data quality and classification accuracy by denoising the ECG signal to remove outliers and noise. Furthermore, our method can reduce data complexity and redundancy through techniques such as dimensionality reduction and regularization, avoiding overfitting or underfitting.

[0085] Feature Extraction: Feature extraction from electrocardiogram (ECG) signals typically relies on manually designed features or rules, such as waveform parameters, frequency domain features, and statistical features. These features may be insufficient to reflect the time-frequency information of the ECG signal or unsuitable for different classification tasks. Our proposed method transforms the ECG signal into a two-dimensional image and uses color to represent amplitude, increasing the dimensionality and information content of the data. Furthermore, our method automatically extracts multi-dimensional features using convolutional layers, avoiding the limitations and complexity of manually designed features or rules.

[0086] Model Training: Training ECG signal models typically requires selecting appropriate network structures, parameter settings, and optimization algorithms. These choices can be influenced by experience or preferences, leading to unstable or unreliable model performance. Our proposed method utilizes transfer learning techniques, borrowing a pre-trained VGG16 model to reduce the difficulty of network structure and parameter settings, thereby improving model performance and reliability. Furthermore, our method can further optimize the model training process through techniques such as cross-validation, grid search, and hyperparameter optimization, enhancing both model effectiveness and efficiency.

[0087] An ECG signal image classification algorithm based on CNN (Convolutional Neural Network) can automatically extract features from images and perform classification. ECG signal imageization is a method of converting ECG signals into images, and this algorithm leverages the advantages of CNN to improve the classification performance of ECG signals.

[0088] This paper presents a CNN-based image classification algorithm for electrocardiogram (ECG) signals, including data preprocessing, network structure, and loss function. Data preprocessing involves normalizing, segmenting, and performing time-frequency transformations on the ECG signals to obtain time-frequency images. Network structure refers to the settings of parameters such as the number of CNN layers, kernel size, and activation function. The loss function is the optimization objective of the CNN; this chapter uses the cross-entropy loss function.

[0089] The experimental results include classification experiments for abnormal heart rate intervals and abnormal heart rate amplitudes. Abnormal heart rate intervals refer to irregularities in the time interval between heartbeats, such as sinus tachycardia and sinus bradycardia. Abnormal heart rate amplitudes refer to irregularities in the amplitude of the electrocardiogram (ECG) waveform, such as ventricular premature beats and atrial premature beats. The experimental results show that the CNN-based ECG signal image classification algorithm can effectively identify these abnormalities and achieves high accuracy and stability. Compared with traditional methods and other time-frequency transformation methods, this method has higher efficiency and accuracy, providing new ideas and methods for clinical medical diagnosis.

[0090] For cases of abnormal intervals, the ECG signal is converted into a color two-dimensional image. Each image contains 10 seconds of signal, divided into 5 rows with 600 points per row, and color represents amplitude. This method preserves the temporal and amplitude information of the ECG signal while utilizing image processing techniques for classification. The image data is divided into two categories: normal and abnormal, with the training and test sets split in a 7:3 ratio, totaling 88 images. The image data is then input into a transfer learning model for training and testing. A transfer learning model refers to using a pre-trained deep learning model and fine-tuning or adjusting it based on a new dataset. This method leverages existing model knowledge to improve classification accuracy and efficiency.

[0091] In this embodiment, the sub-solution first uses wavelet thresholding denoising technology to process the original one-dimensional ECG signal before image conversion and input into the VGG16 model. This improves the signal-to-noise ratio of the input signal, removes noise such as baseline drift, power line interference, and electromyography interference, and lays a high-quality data foundation for subsequent accurate image conversion and feature extraction. This reduces the possibility that the VGG16 model will learn noise features.

[0092] Converting preprocessed one-dimensional electrocardiogram (ECG) signals into two-dimensional image formats serves as a bridge connecting one-dimensional physiological signals with two-dimensional image processing models (VGG16). By mapping the time-domain, frequency-domain, or time-frequency-domain features of the signal to the pixels and colors of the image, powerful convolutional neural networks can effectively extract and learn complex patterns in ECG signals, allowing for more intuitive observation of data characteristics.

[0093] Transfer learning technique: The VGG16 model is not trained from scratch, but rather loaded with pre-trained weights on a large image dataset (such as ImageNet), and then fine-tuned for ECG image data or used as a feature extractor. This significantly reduces the need for large amounts of labeled ECG image data, accelerates model training convergence, and leverages the general image feature extraction capabilities learned by the pre-trained model to improve generalization ability and classification performance on specific tasks such as ECG signals.

[0094] After extracting high-dimensional feature vectors (such as the output of fully connected layers) from the VGG16 model, the PCA algorithm is applied to reduce their dimensionality. This reduces feature redundancy, removes noise, lowers the computational complexity of subsequent classifiers, and may improve classification performance. Furthermore, the dimensionality-reduced features are easier to visualize and analyze.

[0095] The feature vectors extracted by VGG16 and possibly reduced by PCA are input into a newly designed multi-layer fully connected network, and finally the classification result is output through Softmax (or Sigmoid, corresponding to binary classification). The deep features extracted from the image are mapped to a specific classification task (normal / abnormal heart rate) to achieve end-to-end classification.

[0096] Example 2 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0097] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0098] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0099] Example 4 The purpose of this embodiment is to provide a human sensor data processing system based on deep convolutional neural networks, including: The electrocardiogram signal data processing module is configured to: acquire electrocardiogram signal data and classify arrhythmias into two types: amplitude abnormalities and interval abnormalities; and process the electrocardiogram signal data into different color images for different arrhythmias. The feature extraction module is configured to input a color image into the VGG16 model to extract features of different scales and levels of abstraction in the image. The classification module is configured to reduce the dimensionality of the extracted features and classify and visualize them based on the labels.

[0100] Example 5 The purpose of this embodiment is to provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods and functions involved in any of the above embodiments. The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0101] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0102] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A human sensor data processing method based on deep convolutional neural networks, characterized by: include: Acquire electrocardiogram (ECG) signal data and classify arrhythmias into two types: amplitude abnormalities and interval abnormalities. Process the ECG signal data into different color images for different arrhythmias. The color image is input into the VGG16 model to extract features of different scales and levels of abstraction in the image. The extracted features are reduced in dimensionality and then classified and visualized based on the labels.

2. The human sensor data processing method based on deep convolutional neural networks as described in claim 1, characterized in that, For abnormal intervals, the electrocardiogram signal is converted into a color two-dimensional image. The two-dimensional image contains the signal at a set time, divided into multiple rows, with each row having a set number of points. The color represents the amplitude, which is used to preserve the time and amplitude information of the electrocardiogram signal. The two-dimensional images are divided into two categories: normal and abnormal, and the training set and test set are divided according to a set ratio.

3. The human sensor data processing method based on deep convolutional neural networks as described in claim 1, characterized in that, For amplitude abnormalities, the electrocardiogram signal is converted into a time-frequency image. Each time-frequency image contains a signal of one cycle. The horizontal axis represents time, the vertical axis represents frequency, and the color represents amplitude. The frequency domain features of the electrocardiogram signal are extracted based on the time-frequency image to reflect changes in cardiac electrical activity. The time-frequency images are divided into two categories: normal and abnormal. The training set and test set are divided according to a certain ratio. The time-frequency images are then input into the neural network for training and testing.

4. The human sensor data processing method based on deep convolutional neural networks as described in claim 1, characterized in that, The VGG16 model is a convolutional neural network based on transfer learning, which is a deep learning model trained on a large-scale dataset. The convolutional neural network consists of two parts: a feature extraction part and a classifier part; The feature extraction part uses convolutional and pooling layers from the VGG16 model as a convolutional neural network for transfer learning, while the classifier part consists of three fully connected layers and a softmax layer for classification.

5. The human sensor data processing method based on deep convolutional neural networks as described in claim 1, characterized in that, Image features are extracted through convolutional and pooling layers in the VGG16 model. Then, the feature vectors are flattened into one-dimensional vectors by a Flatten layer. These vectors are then mapped through two fully connected layers, each using a ReLU activation function to add non-linearity. After another fully connected layer, the feature vectors are mapped again and then input into a softmax activation function to normalize the vectors into a probability distribution. Finally, a binary cross-entropy loss function is calculated to measure the difference between the predicted results and the true labels.

6. A human sensor data processing system based on deep convolutional neural networks, characterized in that: include: The electrocardiogram signal data processing module is configured to: acquire electrocardiogram signal data and classify arrhythmias into two types: amplitude abnormalities and interval abnormalities; and process the electrocardiogram signal data into different color images for different arrhythmias. The feature extraction module is configured to input a color image into the VGG16 model to extract features of different scales and levels of abstraction in the image. The classification module is configured to reduce the dimensionality of the extracted features and classify and visualize them based on the labels.

7. The human sensor data processing system based on a deep convolutional neural network as described in claim 1, characterized in that, The VGG16 model is a convolutional neural network based on transfer learning, which is a deep learning model trained on a large-scale dataset. The convolutional neural network consists of two parts: a feature extraction part and a classifier part; The feature extraction part uses convolutional and pooling layers from the VGG16 model as a convolutional neural network for transfer learning, while the classifier part consists of three fully connected layers and a softmax layer for classification.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method described in any one of claims 1-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the method described in any one of claims 1-5.

Images