Heart disease prediction method based on face video stream and multi-model cascade

Abstract

The invention discloses a heart disease prediction method based on a face video stream and multi-model cascading, relates to the field of heart disease prediction, and solves the problems of high dependence on the quality of acquisition equipment and the integrity of labeled data, poor robustness and the like in the existing analysis of electrocardiogram data. The method comprises the following steps: positioning and tracking a face region in a video by adopting a face detection method, and preprocessing video frames intercepted by face video stream data in the video by adopting integrogram optimization; the method comprises the following steps: extracting periodic features of a video frame center electric signal by using an autocorrelation function, calculating a time difference between an electrocardiosignal point timestamp and a video frame timestamp, extracting dynamic features of an electrocardiosignal by using a minimum difference method or an interpolation algorithm, and carrying out deep fusion on the dynamic features and space-time dynamic features of a face video stream signal to obtain a face video stream signal. Local dynamic feature distribution of electrocardiosignals is captured through a KNN algorithm, hyper-parameters of the GBDT model are adjusted and optimized through Bayesian optimization, and multi-dimensional performance evaluation is conducted on the GBDT model through cross validation.

Description

Technical Field

[0001] The present invention relates to the technical field of heart disease prediction, and in particular to a heart disease prediction method based on facial video stream and multi-model cascade. Background Art

[0002] With the rapid development of telemedicine and intelligent health monitoring, real-time and accurate prediction of heart disease has become an important research direction in the medical field. Traditional heart disease detection relies on contact medical equipment, such as electrocardiographs (ECGs) and pulse oximeters. Although they are highly accurate, they are highly equipment-dependent, complex to operate, and require professional support, making it difficult to meet the needs of daily health monitoring or large-scale screening.

[0003] In recent years, non-contact detection technologies, such as the combination of facial video streams and deep learning, have opened up new avenues for heart disease prediction. Facial video streams have been applied to real-time monitoring of heart rate and blood circulation by capturing the rhythmic characteristics of blood flow reflected by skin color changes. However, relying solely on facial video streams is difficult to cover the complex pathological features required for heart disease diagnosis, such as abnormal heart rhythm or myocardial damage.

[0004] On the other hand, ECG data, as an authoritative basis for the diagnosis of heart disease, can directly reflect the dynamic process of cardiac electrical activity. However, the analysis of ECG data is highly dependent on the quality of the acquisition equipment and the integrity of the labeled data, and is highly sensitive to noise. The deep learning network has a high computational overhead when processing ECG data, and the model is less robust to small sample data and noisy data. Summary of the invention

[0005] The present invention aims to solve the problems in the prior art that the analysis of ECG data is highly dependent on the quality of the acquisition equipment and the integrity of the labeled data, and is highly sensitive to noise. The deep learning network has a large computational overhead when processing ECG data, and the model has poor robustness to small sample data and noisy data.

[0006] To solve the above technical problems, the present invention is achieved through the following technical solutions: Solution 1: The present invention proposes a method for predicting heart disease based on facial video stream and multi-model cascade, the method comprising the following steps: Step 1: Use the face detection method to locate and track the facial area in the video, extract the color signal containing the heart rate information, and use the integral image optimized CLAHE method to preprocess the facial video stream signal in the video frame; Step 2: using the autocorrelation function to extract the periodic characteristics of the central electrical signal of the video frame in step 1, while the video device records the frame number and the video frame timestamp; Step 3: Calculate the time difference between the ECG signal point timestamp and the video frame timestamp through the timestamp matching algorithm, and use the minimum difference method or interpolation algorithm to achieve accurate matching between the ECG signal and the video frame; Step 4: Extract the dynamic features of the ECG signal through 1D-CNN, and deeply fuse it with the spatiotemporal dynamic features of the facial video stream signal to achieve efficient integration of multimodal data; Step 5: Use the KNN algorithm to capture the local dynamic feature distribution of the ECG signal, automatically tune the hyperparameters of the GBDT model through Bayesian optimization, use cross-validation to perform multi-dimensional performance evaluation of the GBDT model, and complete the prediction of heart disease based on facial video streams and multi-model cascades.

[0007] Further, a preferred implementation is provided, in step 1, a facial detection method is used to locate and track the facial area in the video, and a method for extracting a color signal containing heart rate information is as follows: S.1.1. Use OpenCV library for image and video processing; S.1.2. Initialize the video capture object, use cv2.VideoCapture(0) to open the camera and capture the real-time video stream; S.1.3, load OpenCV's pre-trained Haar feature classifier haarcascade_frontalface_default.xml to detect frontal faces in video frames; S.1.4. Capture live video frames via cap.read() and check the capture status; if the capture is successful, convert the video frames from BGR color space to grayscale images; S.1.5. Use the detectMultiScale function to detect the face area in the grayscale image, mark it with a blue rectangle using cv2.rectangle, and determine the coordinates, color, and border thickness; S.1.6. Take the top quarter of the detected facial rectangle as the area where blood flow changes significantly, extract the color signal in this area, and use it as the input for the next step of signal processing.

[0008] Further, a preferred embodiment is provided, wherein the preprocessing of the video frames intercepted from the facial video stream data in the video in step 1 comprises the following steps: S.1.1.1. Define the compute_integral_histogram(image) function to calculate the integral histogram of the input image image; if the input image is a color image, convert it to a grayscale image through cv2.cvtColor; calculate the integral graph through cv2.integral and store it in the three-dimensional array integral_hist, recording the cumulative sum of each grayscale value at each point; S.1.1.2. Define the fast_histogram_calculation(integral_hist, top_left, bottom_right) function, use the integral histogram to calculate the grayscale histogram in the rectangular area, calculate the number of each grayscale pixel in the rectangular area, and generate the histogram array hist.

[0009] S.1.1.3. Check if the input image is a grayscale image. If it is a color image (image.shape>2), convert it to a grayscale image using cv2.cvtColor and initialize a three-dimensional array integral_hist to store the cumulative sum of all grayscale values. S.1.1.4. Traverse the grayscale value range, generate a binary image for each grayscale value, obtain the binary Boolean image through (gray==i) and convert it to a binary image, then use cv2.integral to calculate its integral image, and store the integral image in the corresponding position of the integral_hist array; S.1.1.5. Define the target rectangular area, use top_left and bottom_right to specify the coordinates of the upper left corner and the lower right corner, and for each gray level in the rectangular area, use the integral histogram to calculate the cumulative sum of the gray values ​​in the area and update the histogram array hist; S.1.1.6. Return the integral histogram integral_hist, where each layer records the cumulative sum of the corresponding grayscale pixels; S.1.1.7, apply CLAHE equalization processing to the grayscale histogram hist of a given area, map the equalized histogram back to the original area, and adjust the grayscale values ​​of pixels in the area; S.1.1.8. Merge all the small blocks after equalization to obtain the complete enhanced image enhanced_image.

[0010] Further, a preferred embodiment is provided, in which the method for extracting the periodic characteristics of the central electrical signal of the video frame by the autocorrelation function in step 2 comprises the following steps: S2.1. Import the required libraries, use numpy for mathematical operations and array manipulation, and use matplotlib.pyplot to plot the charts of the signals and analysis results. S2.2. Define the generate_signal function to generate a simulation signal: use np.linspace to generate a time vector t, create a sinusoidal signal, add the average value that changes linearly with time, and return the time vector t and the generated signal signal; S2.3. Define the autocorrelation function to calculate the autocorrelation of the signal: decentralize the signal; S2.4. Define the dynamic_segment_analysis function to implement dynamic segmented autocorrelation: input signal signal, sampling rate fs, segment length segment_length and step length step; initialize the starting point start and cyclically segment the signal, each segment length is segment_length; calculate the autocorrelation result for each segment of the signal and analyze the dynamic signal changes; S2.5. Define the estimate_period function to calculate the signal period: use scipy.signal.find_peaks to detect all significant peaks in the autocorrelation results; check whether at least two peaks are detected, calculate the distance between adjacent peaks by np.diff, and take the average as the period estimate; divide the average period by the sampling frequency fs and convert to seconds; S2.6. Calculate heart rate based on cycles: define the heart rate as the number of cycles per minute; if the maximum peak is detected, return the estimated heart rate, otherwise return None, indicating that the estimation failed; S2.7, combining dynamic segment analysis with heart rate estimation: cyclically moving the starting point start, gradually processing the entire signal, and calculating the heart rate of each segment of the signal in real time; S2.8. Set the signal length to 10 seconds, the sampling rate to 100 Hz, and the signal frequency to 1 Hz. Perform the above steps on the simulated heart rate signal and calculate the heart rate using autocorrelation and peak detection.

[0011] Furthermore, a preferred implementation is provided, in which the data after the precise matching of the ECG signal and the video frame in step 3 includes the ECG signal timestamp, ECG value, heart rate value, video frame sequence number and timestamp, and a unified timeline joint data table is generated.

[0012] Furthermore, a preferred implementation is provided, in which the spatiotemporal dynamic features of the facial video signal in step 4 are extracted by using a CNN algorithm and a LSTM algorithm.

[0013] Furthermore, a preferred implementation is provided, wherein step 5 also includes the steps of normalizing and one-hot encoding the dynamic features.

[0014] Furthermore, a preferred embodiment is provided, wherein step 5 also includes the steps of cleaning and preprocessing the heart disease prediction data set.

[0015] Solution 3: A computer device includes a processor and a storage medium, wherein the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes any one of the methods described in Solution 1.

[0016] Solution 4: A computer storage medium for storing a computer program. When the computer program is read by a computer, the computer executes any one of the methods described in Solution 1.

[0017] The present invention is beneficial in that: The heart disease prediction method based on facial video stream and multi-model cascade described in the present invention uses the innovative design of multimodal fusion and multi-model cascade, and integrates facial video stream and ECG data to construct a multimodal data fusion framework and a multi-model cascade prediction method to improve the accuracy and robustness of heart disease prediction. The dynamic features of multimodal data are extracted by optimizing the facial video stream signal and synchronizing the timestamp of the ECG signal, and feature integration is performed through deep learning models such as 1D-CNN and LSTM. Combining the cascade modeling and Bayesian optimization of KNN and GBDT models, the ability to capture complex feature relationships is further improved, and ultimately efficient prediction of heart disease is achieved, achieving high efficiency, accuracy and robustness in heart disease prediction, providing an efficient and robust heart disease prediction solution, and providing important technical support for telemedicine and intelligent health monitoring.

[0018] The present invention is also applicable to the field of combining biological signal processing, computer vision and deep learning technology, that is, belonging to the field of intelligent health monitoring and telemedicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a method for predicting heart disease based on facial video stream and multi-model cascade as described in Implementation Method 1.

[0020] Figure 2 This is a schematic diagram of the measurement results of cardiovascular health parameters of different subjects described in Implementation Example 11.

[0021] Figure 2 In the figure, (a) is a schematic diagram showing the change of the accuracy of video stream facial data collection over time; (b) is a schematic diagram showing the change of the accuracy of video stream combined with GBDT method over time, and (c) is a schematic diagram showing the change of the accuracy of GBDT method alone over time. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the implementation methods of the present application clearer, the technical solutions in the implementation methods of the present application will be clearly and completely described below in conjunction with the drawings in the implementation methods of the present application. Obviously, the described implementation methods are only part of the implementation methods of the present application, not all of the implementation methods.

[0023] Embodiment 1: This embodiment provides a method for predicting heart disease based on facial video stream and multi-model cascade, the method comprising the following steps: Step 1: Use the face detection method to locate and track the facial area in the video, extract the color signal containing the heart rate information, and use the integral image optimized CLAHE method to preprocess the facial video stream signal in the video frame; Step 2: using the autocorrelation function to extract the periodic characteristics of the central electrical signal of the video frame in step 1, while the video device records the frame number and the video frame timestamp; Step 3: Calculate the time difference between the ECG signal point timestamp and the video frame timestamp through the timestamp matching algorithm, and use the minimum difference method or interpolation algorithm to achieve accurate matching between the ECG signal and the video frame; Step 4: Extract the dynamic features of the ECG signal through 1D-CNN, and deeply fuse it with the spatiotemporal dynamic features of the facial video stream signal to achieve efficient integration of multimodal data; Step 5: Use the KNN algorithm to capture the local dynamic feature distribution of the ECG signal, automatically tune the hyperparameters of the GBDT model through Bayesian optimization, use cross-validation to perform multi-dimensional performance evaluation of the GBDT model, and complete the prediction of heart disease based on facial video streams and multi-model cascades.

[0024] Embodiment 2: This embodiment further limits the heart disease prediction method based on facial video stream and multi-model cascade described in embodiment 1. In step 1, a facial detection method is used to locate and track the facial area in the video, and a method for extracting a color signal containing heart rate information is as follows: S.1.1. Use OpenCV library for image and video processing; S.1.2. Initialize the video capture object, use cv2.VideoCapture(0) to open the camera and capture the real-time video stream; S.1.3, load OpenCV's pre-trained Haar feature classifier haarcascade_frontalface_default.xml to detect frontal faces in video frames; S.1.4. Capture live video frames via cap.read() and check the capture status; if the capture is successful, convert the video frames from BGR color space to grayscale images; S.1.5. Use the detectMultiScale function to detect the face area in the grayscale image, mark it with a blue rectangle using cv2.rectangle, and determine the coordinates, color, and border thickness; S.1.6. Take the top quarter of the detected facial rectangle as the area where blood flow changes significantly, extract the color signal in this area, and use it as the input for the next step of signal processing.

[0025] Embodiment 3: This embodiment further limits the heart disease prediction method based on facial video stream and multi-model cascade described in embodiment 1. In step 1, preprocessing the video frames intercepted from the facial video stream data in the video includes the following steps: S.1.1.1. Define the compute_integral_histogram(image) function to calculate the integral histogram of the input image image; if the input image is a color image, convert it to a grayscale image through cv2.cvtColor; calculate the integral graph through cv2.integral and store it in the three-dimensional array integral_hist, recording the cumulative sum of each grayscale value at each point; S.1.1.2. Define the fast_histogram_calculation(integral_hist, top_left, bottom_right) function, use the integral histogram to calculate the grayscale histogram in the rectangular area, calculate the number of each grayscale pixel in the rectangular area, and generate the histogram array hist.

[0026] S.1.1.3. Check if the input image is a grayscale image. If it is a color image (image.shape>2), convert it to a grayscale image using cv2.cvtColor and initialize a three-dimensional array integral_hist to store the cumulative sum of all grayscale values. S.1.1.4. Traverse the grayscale value range, generate a binary image for each grayscale value, obtain the binary Boolean image through (gray==i) and convert it to a binary image, then use cv2.integral to calculate its integral image, and store the integral image in the corresponding position of the integral_hist array; S.1.1.5. Define the target rectangular area, use top_left and bottom_right to specify the coordinates of the upper left corner and the lower right corner, and for each gray level in the rectangular area, use the integral histogram to calculate the cumulative sum of the gray values ​​in the area and update the histogram array hist; S.1.1.6. Return the integral histogram integral_hist, where each layer records the cumulative sum of the corresponding grayscale pixels; S.1.1.7, apply CLAHE equalization processing to the grayscale histogram hist of a given area, map the equalized histogram back to the original area, and adjust the grayscale values ​​of pixels in the area; S.1.1.8. Merge all the small blocks after equalization to obtain the complete enhanced image enhanced_image.

[0027] Embodiment 4: This embodiment further limits the heart disease prediction method based on facial video stream and multi-model cascade described in embodiment 1. The method of extracting the periodic characteristics of the central electrical signal of the video frame by the autocorrelation function in step 2 includes the following steps: S2.1. Import the required libraries, use numpy for mathematical operations and array manipulation, and use matplotlib.pyplot to plot the charts of the signals and analysis results. S2.2. Define the generate_signal function to generate a simulation signal: use np.linspace to generate a time vector t, create a sinusoidal signal, add the average value that changes linearly with time, and return the time vector t and the generated signal signal; S2.3. Define the autocorrelation function to calculate the autocorrelation of the signal: decentralize the signal; S2.4. Define the dynamic_segment_analysis function to implement dynamic segmented autocorrelation: input signal signal, sampling rate fs, segment length segment_length and step length step; initialize the starting point start and cyclically segment the signal, each segment length is segment_length; calculate the autocorrelation result for each segment of the signal and analyze the dynamic signal changes; S2.5. Define the estimate_period function to calculate the signal period: use scipy.signal.find_peaks to detect all significant peaks in the autocorrelation results; check whether at least two peaks are detected, calculate the distance between adjacent peaks by np.diff, and take the average as the period estimate; divide the average period by the sampling frequency fs and convert to seconds; S2.6. Calculate heart rate based on cycles: define the heart rate as the number of cycles per minute; if the maximum peak is detected, return the estimated heart rate, otherwise return None, indicating that the estimation failed; S2.7, combining dynamic segment analysis with heart rate estimation, cyclically moving the starting point start, gradually processing the entire signal, and calculating the heart rate of each segment of the signal in real time; S2.8. Set the signal length to 10 seconds, the sampling rate to 100 Hz, and the signal frequency to 1 Hz. Perform the above steps on the simulated heart rate signal and calculate the heart rate using autocorrelation and peak detection.

[0028] Implementation method five: This implementation method further limits the heart disease prediction method based on facial video stream and multi-model cascade described in implementation method one. In step 3, the data after the precise matching of the ECG signal and the video frame includes the ECG signal timestamp, ECG value, heart rate value, video frame number and timestamp, and generates a unified timeline joint data table.

[0029] Implementation method 6: This implementation method further limits the heart disease prediction method based on facial video stream and multi-model cascade described in implementation method 1. In step 4, the spatiotemporal dynamic features of the facial video signal are extracted by CNN algorithm and LSTM algorithm.

[0030] Embodiment 7: This embodiment further limits the heart disease prediction method based on facial video stream and multi-model cascade described in embodiment 1, and step 5 also includes the steps of normalizing and one-hot encoding the dynamic features.

[0031] Embodiment 8: This embodiment further limits the heart disease prediction method based on facial video stream and multi-model cascade described in embodiment 1, and step 5 also includes the steps of cleaning and preprocessing the heart disease prediction data set.

[0032] Embodiment 9. This embodiment proposes a computer device, including a processor and a storage medium, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the method described in any one of embodiments 1 to 8.

[0033] Embodiment 10: This embodiment proposes a computer storage medium for storing a computer program. When the computer program is read by a computer, the computer executes the method described in any one of Embodiments 1 to 8.

[0034] Implementation 11. This implementation includes an example, which is used to explain the above implementation. Figure 1 to Figure 2 The present embodiment is described. The heart disease prediction method based on facial video stream and multi-model cascade described in the present embodiment specifically comprises the following steps: (1) Use facial detection technology to locate and track the facial area in the video, focusing on the areas where blood flow changes significantly (such as the forehead), and extract the color signal containing heart rate information. Preprocess the video frame through local contrast adjustment with integral image enhancement, significantly improve the local contrast of the image, make the slight changes in skin color caused by blood circulation more prominent, and accelerate the calculation of local histograms in CLAHE to enhance the adaptability to complex lighting and shadow interference. Use the optimized autocorrelation function instead of the traditional Butterworth filter to process the extracted signal. The autocorrelation function analyzes the periodicity of the signal and identifies significant peaks. The video data contains the frame number and frame timestamp.

[0035] (2) When collecting ECG signals, reserve timestamps, match the timestamps of each ECG data point, find the closest video frame timestamp, integrate the matched ECG data with the video frame information, and form a joint data containing ECG timestamps, ECG values, and heart rate values. Use the KNN algorithm to fill in missing values ​​of the joint data to ensure data consistency; use KNN to capture local feature distributions, and combine the nonlinear modeling capabilities of GBDT to optimize prediction performance; introduce a deep learning network to extract the dynamic spatiotemporal features and ECG waveform features of facial video streams, and achieve accurate characterization of complex disease features through multimodal feature fusion. In addition, Bayesian optimization is used to dynamically tune model parameters, and the model performance is quantified through a multi-model evaluation framework to ensure the robustness and applicability of the system.

[0036] (1) Facial region localization and signal extraction: This implementation method accurately captures the color signal of the forehead area through positioning and signal extraction of the facial area, providing high-quality and stable input data for subsequent heart rate signal analysis and processing, while ensuring real-time and robustness.

[0037] Step 1. Import necessary libraries and use the OpenCV library (cv2) for image and video processing. The NumPy library (numpy) supports matrix operations and processing of image data.

[0038] Step 2: Initialize the video capture object and use cv2.VideoCapture(0) to open the default camera and capture the real-time video stream.

[0039] Step 3: Load OpenCV's pre-trained Haar feature classifier haarcascade_frontalface_default.xml to detect frontal faces in video frames.

[0040] Step 4: Capture the live video frame through cap.read() and check the capture status. If the capture is successful, convert the frame from BGR color space to grayscale image to reduce the computational complexity and provide efficient support for face detection.

[0041] Step 5. Use the detectMultiScale function to detect the face area in the grayscale image. The parameters include: scaling factor: 1.1 (reducing the image scale layer by layer). Minimum number of neighbors: 4 (the number of neighbors required for the current rectangle). The detected face area is marked with a blue rectangle using cv2.rectangle, and the coordinates, color, and border thickness are specified.

[0042] Step 6: Take the top quarter of the detected facial rectangle as the forehead area, extract the color signal in the area, and use it as input for subsequent signal processing.

[0043] (2) Video frame preprocessing: Through video frame preprocessing, the regional grayscale histogram is extracted using integral histogram and fast calculation technology, and combined with CLAHE equalization processing, the local contrast of the image is enhanced, the grayscale information is highlighted, and the accuracy of subsequent feature extraction is improved.

[0044] Step 1. Define the compute_integral_histogram(image) function to calculate the integral histogram of the input image image. If the input image is a color image, convert it to a grayscale image through cv2.cvtColor. Generate a binary image for each grayscale value (0~255), calculate the integral image through cv2.integral, and store it in the three-dimensional array integral_hist, recording the cumulative sum of each grayscale value at each point.

[0045] Step 2, define the fast_histogram_calculation(integral_hist, top_left, bottom_right) function, and use the integral histogram to quickly calculate the grayscale histogram in the rectangular area. Through the addition and subtraction properties of the integral histogram, calculate the number of each grayscale pixel in the specified area and generate the histogram array hist.

[0046] Step 3. Check if the input image is a grayscale image. If it is a color image (image.shape>2), convert it to a grayscale image using cv2.cvtColor. Initialize a three-dimensional array integral_hist with a size of image height * width * grayscale (256) to store the cumulative sum of all grayscale values.

[0047] Step 4: Traverse the grayscale value range (0~255), generate a binary image for each grayscale value, obtain the binary Boolean image through (gray==i) and convert it to a binary image, and then use cv2.integral to calculate its integral image. Store the integral image in the corresponding position of the integral_hist array.

[0048] Step 5: Define the target rectangular area, and use top_left and bottom_right to specify the coordinates of the upper left corner and the lower right corner. For each gray level in the rectangular area, use the integral histogram to calculate the cumulative sum of the gray values ​​in the area and update the histogram array hist.

[0049] Step 6: Return the integral histogram integral_hist, where each layer records the cumulative sum of the corresponding grayscale pixels.

[0050] Step 7: Apply CLAHE equalization to the grayscale histogram of the given area, and avoid noise amplification by limiting contrast enhancement (clip_limit parameter). Map the equalized histogram back to the original area and adjust the grayscale value of the pixels in the area.

[0051] Step 8: Merge all the equalized small blocks to obtain the complete enhanced image enhanced_image to improve the overall contrast.

[0052] (3) Signal processing and heart rate estimation: This embodiment uses signal processing and heart rate estimation, uses autocorrelation analysis to identify signal periodic characteristics, and calculates heart rate through peak detection, thereby providing accurate time and frequency characteristics for non-contact heart rate monitoring.

[0053] Step 1. Import the required libraries and use numpy for mathematical operations and array operations. Use matplotlib.pyplot to plot the signal and analysis results.

[0054] Step 2. Define the generate_signal function to generate an analog signal: the input parameters include the signal length (seconds), the sampling rate fs (number of samples per second), and the signal frequency freq (Hz). Use np.linspace to generate the time vector t, create a sinusoidal signal, and add the average value that changes linearly with time, and return the time vector t and the generated signal signal.

[0055] Step 3. Define the autocorrelation function to calculate the autocorrelation of the signal: Decenter the signal (subtract its mean) to ensure that the autocorrelation result is accurate. Use np.correlate to calculate the autocorrelation and select the part from the midpoint to the end. Normalize the autocorrelation result so that its maximum value is 1 for subsequent processing.

[0056] Step 4. Define the dynamic_segment_analysis function to implement dynamic segmented autocorrelation: input signal signal, sampling rate fs, segment length segment_length and step length step. Initialize the starting point start and cyclically segment the signal, each segment length is segment_length. Calculate the autocorrelation result for each segment of the signal and analyze the dynamic signal changes.

[0057] Step 5. Define the estimate_period function to calculate the signal period: Use scipy.signal.find_peaks to detect all significant peaks (height > 0.5) in the autocorrelation results. Check whether at least two peaks are detected to ensure that the period can be calculated. Calculate the distance between adjacent peaks through np.diff and take the average as the period estimate (in units of sampling points). Divide the average period by the sampling frequency fs and convert to seconds.

[0058] Step 6. Calculate the heart rate based on the cycle. Define the heart rate as the number of cycles per minute and use the formula 60 / average cycle time to calculate. If enough peaks are detected, the estimated heart rate is returned, otherwise None is returned, indicating that the estimation failed.

[0059] Step 7: Combine dynamic segment analysis with heart rate estimation: cyclically move the starting point start, gradually process the entire signal, and calculate the heart rate of each signal segment in real time.

[0060] Step 8: Sample signal processing, set the signal length to 10 seconds, the sampling rate to 100 Hz, and the signal frequency to 1 Hz. Perform the above steps on the simulated heart rate signal and calculate the heart rate using autocorrelation and peak detection.

[0061] Step 9: Verify the accuracy of the heart rate estimation, plot the signal and its autocorrelation function, and visually display the periodic characteristics. Analyze the effectiveness of peak detection and verify whether the heart rate estimation conforms to the periodic characteristics of the signal.

[0062] (4) Joint data cleaning and preprocessing Data cleaning, cleaning and preprocessing of heart disease prediction datasets to provide high-quality input data for subsequent modeling.

[0063] Step 1: Import necessary libraries and set up the environment. Import pandas and numpy libraries for data processing and numerical operations.

[0064] Step 2: Define the one-hot encoding function. Define the function One_hot to convert the numerical discrete features (such as sex and cp) in the data set into string labels with practical meanings. After converting discrete features (such as gender and chest pain type) into string labels with practical meanings, use one-hot encoding to expand them into multiple feature columns to provide a computationally friendly input format for the model.

[0065] Step 3: Define the normalization function. Define the function Min_max_normalize. Use the minimum-maximum normalization formula to compress the numerical features to the [0, 1] interval to ensure that the feature dimensions are consistent and improve the model performance.

[0066] Step 4: Load the data and apply preprocessing. Use pandas.read_csv to load the original dataset. Call the One_hot function to perform one-hot encoding on the discrete features in the data. Call the Min_max_normalize function to normalize the encoded data.

[0067] Step 5. Save the processed data as a new CSV file named Min_Max_One_hot_KNN_process_hungarian_heart_all.csv.

[0068] Data preprocessing: This implementation provides high-quality input data for the heart disease prediction model after cleaning and normalizing the joint data set. First, all features in the data are converted to numerical types through unified loading and processing steps, and the missing values ​​are intelligently filled using the KNN algorithm to ensure data integrity. Secondly, the target column target is binarized (1 represents a heart disease patient, 0 represents a healthy individual) and the format is unified to meet the needs of the classification model. At the same time, redundant features are screened and removed (such as slope, ca, etc. in Hungarian data), the data structure is optimized, and the interference of irrelevant variables on the model is reduced.

[0069] Step 1: Load the dataset and define column names. Use the pandas library to load the Cleveland and Hungarian datasets and name the feature columns (such as age, gender, angina type, etc.) and the target column target. Convert all columns to numeric types. Unparsed values ​​are set to NaN to mark them as missing values.

[0070] Step 2: Feature processing. Cleveland data: For feature thal, remap the discrete values ​​3, 6, and 7 to 0, 1, and 2, respectively. Hungarian data: Delete redundant columns (such as slope, ca, and thal) to reduce the interference of irrelevant information on modeling.

[0071] Step 3: Binarize the target column. Process the target column target and map all values ​​greater than 0 to 1 (heart disease patients) and otherwise to 0 (healthy individuals).

[0072] Step 4: Fill missing values ​​using KNNImputer: Cleveland data: For specific features (such as ca and thal), fill missing values ​​using the nearest neighbor algorithm with a K value of 3. Hungarian data: Globally fill missing values ​​for the entire data frame to ensure completeness.

[0073] Step 5: Discrete value conversion: convert discrete features (such as fbs, restecg, exang) to integer types to adapt to subsequent machine learning models.

[0074] Step 6. Save the processing results and save the preprocessed data as KNN_process_cleveland_heart.csv and KNN_process_hungarian_heart.csv respectively to provide clean and consistent data input for subsequent heart disease prediction modeling.

[0075] (5) Model prediction Step 1: Data loading and feature / target separation, use pandas.read_csv to load the preprocessed heart disease dataset Min_Max_One_hot_KNN_process_hungarian_heart.csv. Divide the data into feature columns and target columns, with the features stored in the variable x and the target variable stored in the target column in y.

[0076] Step 2. Define the hyperparameter search space and set the hyperparameter search range pbounds of the GBDT model: n_estimators: the number of trees (50-200). min_samples_split: the minimum number of split samples (2-10). max_features: the maximum feature ratio (0.1-0.99). max_depth: the maximum depth of the tree (3-8). min_samples_leaf: the minimum number of leaf node samples (1-13).

[0077] learning_rate: learning rate range (0.01-0.05). random_state: random seed range (20-50).

[0078] Step 3: Model evaluation function GBDT_evaluate, define the evaluation function GBDT_evaluate, which is used to calculate the performance of the GBDT model: Use 10-fold cross validation (KFold) to divide the data. Use cross_val_score to evaluate the GBDT model with different hyperparameter combinations. Use the mean cross-validation accuracy as the objective function value.

[0079] Step 4. Load the preprocessed heart disease dataset, store the feature column in the variable x, and the target column in y for subsequent model training and evaluation. In the hyperparameter optimization stage, Bayesian optimization tunes multiple key hyperparameters of the GBDT model, including: n_estimators (number of decision trees): ranges from 50 to 200, controlling the complexity and fitting ability of the model. min_samples_split (minimum number of split samples): ranges from 2 to 10, determines the minimum number of samples required for each node to split, and prevents overfitting. max_features (maximum feature ratio): ranges from 0.1 to 0.99, controls the number of features considered at each split, and affects the generalization ability of the model. max_depth (maximum depth of the tree): ranges from 3 to 8, limiting the depth of the tree to prevent overfitting. min_samples_leaf (minimum number of leaf node samples): ranges from 1 to 13, used to control the minimum number of samples in the leaf node, balancing model complexity and performance. learning_rate (learning rate): ranges from 0.01 to 0.05, determines the contribution weight of each tree to the overall model, and adjusts the model update step size. random_state (random seed): ranges from 20 to 50, used to ensure the repeatability of the results. Bayesian optimization uses the objective function GBDT_evaluate and the average accuracy of 10-fold cross validation as the optimization indicator to dynamically search for the best combination of these hyperparameters. In the initial stage, 5 groups of parameters are randomly selected for evaluation (init_points=5), and then the acquisition function EI (expected improvement) is used to iterate 50 times in the parameter space (n_iter=50) to gradually approach the global optimal solution. Output the optimal parameter combination params_max and return it.

[0080] Step 5, model training and evaluation function GBDT, use GradientBoostingClassifier to initialize the GBDT model, and pass in the optimized hyperparameters. Use train_test_split to divide the data into training set and test set, with the test set ratio being 20%. Call the fit method on the training set to train the model. Call the predict and predict_proba methods on the test set to make predictions and calculate the performance indicators: accuracy: training set and test set accuracy, confusion matrix: detailed distribution of classification results, recall and precision, ROC curve and its AUC value.

[0081] It can be seen that the present invention can flexibly adapt to high-precision real-time heart rate monitoring of facial video streams through the heart disease prediction method based on facial video streams and multi-model cascade.

[0082] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0083] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A heart disease prediction method based on facial video stream and multi-model cascade, characterized in that: The method comprises the following steps: Step 1: Use the face detection method to locate and track the facial area in the video, extract the color signal containing the heart rate information, and use the integral image optimized CLAHE method to preprocess the facial video stream signal in the video frame; Step 2: using the autocorrelation function to extract the periodic characteristics of the central electrical signal of the video frame in step 1, while the video device records the frame number and the video frame timestamp; Step 3: Calculate the time difference between the ECG signal point timestamp and the video frame timestamp through the timestamp matching algorithm, and use the minimum difference method or interpolation algorithm to achieve accurate matching between the ECG signal and the video frame; Step 4: Extract the dynamic features of the ECG signal through 1D-CNN, and deeply fuse it with the spatiotemporal dynamic features of the facial video stream signal to achieve efficient integration of multimodal data; Step 5: Use the KNN algorithm to capture the local dynamic feature distribution of the ECG signal, automatically tune the hyperparameters of the GBDT model through Bayesian optimization, use cross-validation to perform multi-dimensional performance evaluation of the GBDT model, and complete the prediction of heart disease based on facial video streams and multi-model cascades.

2. The heart disease prediction method based on facial video stream and multi-model cascade according to claim 1 is characterized in that: In step 1, the facial detection method is used to locate and track the facial area in the video, and the method for extracting the color signal containing the heart rate information is as follows: S.1.

1. Use OpenCV library for image and video processing; S.1.

2. Initialize the video capture object, use cv2.VideoCapture(0) to open the camera and capture the real-time video stream; S.1.3, load OpenCV's pre-trained Haar feature classifier haarcascade_frontalface_default.xml to detect frontal faces in video frames; S.1.

4. Capture live video frames via cap.read() and check the capture status; if the capture is successful, convert the video frames from BGR color space to grayscale images; S.1.

5. Use the detectMultiScale function to detect the face area in the grayscale image, mark it with a blue rectangle using cv2.rectangle, and determine the coordinates, color, and border thickness; S.1.

6. Take the top quarter of the detected facial rectangle as the area where blood flow changes significantly, extract the color signal in this area, and use it as input for the next step of signal processing.

3. The heart disease prediction method based on facial video stream and multi-model cascade according to claim 1, characterized in that: The preprocessing of the video frames captured from the facial video stream data in the video in step 1 includes the following steps: S.1.1.

1. Define the compute_integral_histogram(image) function to calculate the integral histogram of the input image image; if the input image is a color image, convert it to a grayscale image through cv2.cvtColor; calculate the integral graph through cv2.integral and store it in the three-dimensional array integral_hist, recording the cumulative sum of each grayscale value at each point; S.1.1.

2. Define the fast_histogram_calculation(integral_hist, top_left, bottom_right) function, use the integral histogram to calculate the grayscale histogram in the rectangular area, calculate the number of each grayscale pixel in the rectangular area, and generate the histogram array histogram; S.1.1.

3. Check if the input image is a grayscale image. If it is a color image (image.shape>2), convert it to a grayscale image using cv2.cvtColor and initialize a three-dimensional array integral_hist to store the cumulative sum of all grayscale values. S.1.1.

4. Traverse the grayscale value range, generate a binary image for each grayscale value, obtain the binary Boolean image through (gray==i) and convert it to a binary image, then use cv2.integral to calculate its integral image, and store the integral image in the corresponding position of the integral_hist array; S.1.1.

5. Define the target rectangular area, use top_left and bottom_right to specify the coordinates of the upper left corner and the lower right corner, and for each gray level in the rectangular area, use the integral histogram to calculate the cumulative sum of the gray values ​​in the area and update the histogram array hist; S.1.1.

6. Return the integral histogram integral_hist, where each layer records the cumulative sum of the corresponding grayscale pixels; S.1.1.7, apply CLAHE equalization processing to the grayscale histogram hist of a given area, map the equalized histogram back to the original area, and adjust the grayscale values ​​of pixels in the area; S.1.1.

8. Merge all the small blocks after equalization to obtain the complete enhanced image enhanced_image.

4. The heart disease prediction method based on facial video stream and multi-model cascade according to claim 1, characterized in that: The method for extracting the periodic characteristics of the central electric signal of the video frame by the autocorrelation function in step 2 comprises the following steps: S2.

1. Import the required libraries, use numpy for mathematical operations and array manipulation, and use matplotlib.pyplot to plot charts of signals and analysis results. S2.

2. Define the generate_signal function to generate a simulation signal, use np.linspace to generate a time vector t, create a sinusoidal signal, add the average value that changes linearly with time, and return the time vector t and the generated signal signal; S2.3, define the autocorrelation function to calculate the autocorrelation of the signal and decentralize the signal; S2.

4. Define the dynamic_segment_analysis function to implement dynamic segmented autocorrelation, input signal signal, sampling rate fs, segment length segment_length and step length step; initialize the starting point start and cyclically segment the signal, each segment length is segment_length; calculate the autocorrelation result for each segment signal and analyze the dynamic signal changes; S2.

5. Define the estimate_period function to calculate the signal period, use scipy.signal.find_peaks to detect all significant peaks in the autocorrelation results; check whether at least two peaks are detected, calculate the distance between adjacent peaks by np.diff, and take the average as the period estimate; divide the average period by the sampling frequency fs and convert to seconds; S2.

6. Calculate the heart rate based on the cycle, defining the heart rate as the number of cycles per minute; if the maximum peak is detected, return the estimated heart rate, otherwise return None, indicating that the estimation failed; S2.7, combining dynamic segment analysis with heart rate estimation: cyclically moving the starting point start, gradually processing the entire signal, and calculating the heart rate of each segment of the signal in real time; S2.

8. Set the signal length to 10 seconds, the sampling rate to 100 Hz, and the signal frequency to 1 Hz. Perform the above steps on the simulated heart rate signal and calculate the heart rate using autocorrelation and peak detection.

5. The heart disease prediction method based on facial video stream and multi-model cascade according to claim 1, characterized in that: In step 3, the data after the accurate matching of the ECG signal and the video frame includes the ECG signal timestamp, ECG value, heart rate value, video frame sequence number and timestamp, and generates a unified time axis joint data table.

6. The heart disease prediction method based on facial video stream and multi-model cascade according to claim 1, characterized in that: In step 4, the spatiotemporal dynamic features of the facial video signal are extracted through the CNN algorithm and the LSTM algorithm.

7. The heart disease prediction method based on facial video stream and multi-model cascade according to claim 1, characterized in that: Step 5 also includes the steps of normalizing and one-hot encoding the dynamic features.

8. The heart disease prediction method based on facial video stream and multi-model cascade according to claim 1, characterized in that: Step 5 also includes the steps of cleaning and preprocessing the heart disease prediction dataset.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images