Method for collecting finger pulse health data based on mobile terminal mobile phone camera

Abstract

The invention provides a method for collecting finger pulse health data based on a mobile phone camera of a mobile terminal, which comprises the following steps: S1, collecting a video, S2, preprocessing a photoelectric volume pulse wave PPG signal, S3, extracting an original photoelectric volume pulse wave PPG signal, S4, denoising the pulse wave signal, S5, averaging a single-cycle pulse wave, S6, calculating a pulse condition signal threshold-free recurrence plot, and S5, obtaining the finger pulse health data based on the pulse condition signal threshold-free recurrence plot. S7, establishing a pulse condition signal ensemble classification model; and S8, outputting a pulse condition classification analysis result. According to the method, the pulse oscillogram with high characteristic signals is obtained through deep algorithm research, in-depth systematic analysis is carried out on the pulse oscillogram in combination with medical theoretical knowledge, and effective and high-accuracy human health data analysis is established.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence health big data technology, and in particular to a method for collecting finger pulse health data based on a mobile terminal mobile phone camera. Background Technology

[0002] Pulse, as an important physiological indicator, often requires measurement by specialized instruments. However, these instruments are not portable and cannot be used for real-time measurement in certain situations. Furthermore, pulse reading and analysis require interpretation by professional physicians, which consumes significant medical resources. Since high-quality medical resources are scarce, simple pulse testing instruments are not suitable for the general public.

[0003] With the development of mobile communication technology and the continuous improvement of people's living standards, the use of various mobile terminals such as mobile phones is becoming more and more widespread. Mobile phones have become an indispensable communication tool in people's lives. However, existing mobile phones basically do not provide in-depth pulse health interpretation, analysis and health advice based on pulse waveform diagrams for the collected pulse signals.

[0004] Therefore, developing a convenient and quick technology that can detect human health data and monitor health at any time using a mobile phone camera, like having a caring family doctor on hand to provide health reminders, is a technology of great social significance. Summary of the Invention

[0005] This invention provides a method for collecting finger pulse health data based on a mobile phone camera. Its main purpose is to obtain pulse waveforms with high-feature signals through deep learning algorithms, and to conduct in-depth systematic analysis of the pulse waveforms in combination with medical theoretical knowledge, so as to establish a set of effective and accurate human health data analysis methods.

[0006] This invention provides a method for collecting finger pulse health data based on a mobile terminal's mobile phone camera, comprising the following steps: S1 captures video. S2 photoplethysmography (PPG) signal preprocessing S3 extracts the raw PPG signal from the photoplethysmography (PPG) wave. S4 pulse wave signal denoising processing S5 average single-cycle pulse wave S6 pulse signal threshold-free recursive plot S7 establishes an integrated classification model for pulse signals. Output of S8 pulse classification analysis results.

[0007] The video acquisition method described in S1 uses a mobile phone camera to capture the photoplethysmography (PPG) signal of the index fingertip. The sampling frequency is 20-200Hz, and the video is recorded at a speed of 20-60 frames per second for a duration of 20-120 seconds.

[0008] The S2 photoplethysmography (PPG) signal preprocessing module needs to convert the camera's YUV format data every 50 milliseconds, convert the numerical matrix into PPG signal values ​​according to the PPG signal value calculation algorithm, and perform smoothing and noise reduction processing on the continuously generated PPG datasets.

[0009] The YUV format conversion mentioned above involves calling the onPreviewFrame method in Android during Camera preview. The variable data stores the YUV format byte matrix of the current video frame. During processing, this variable data needs to be converted from YUV format to RGB format.

[0010] Furthermore, there is a fixed conversion formula for the mutual conversion between YUV and RGB formats. However, the coefficients in the formula are all in non-integer form, which involves a large number of floating-point operations in the programming implementation, which will seriously affect the performance of the conversion algorithm. This invention provides an efficient YUV conversion method that transforms the performance-intensive floating-point operations into extremely fast bitwise operations, thereby improving the performance of the conversion algorithm.

[0011] Furthermore, the improved algorithm formula for YUV to RGB conversion is as follows:

[0012] The simplified formula is as follows:

[0013] The formula, after further optimization, is as follows:

[0014] By replacing all multiplication operations with bit shifting operations, the above formula can be easily optimized using bit shifting, as shown below:

[0015] In computer systems, addition, subtraction, and shifting algorithms are far more efficient than multiplication. Therefore, multiplication needs to be completely replaced, and the algorithm needs further optimization, as shown in the following formula:

[0016] The S3 process extracts the raw PPG signal. In the RGB color model, the fluctuation trend of the red channel most closely matches the fluctuation trend of the PPG signal value curve; therefore, the red channel value X from the video stream data is selected. r As a calculation variable, the average red value R of each pixel in the two-dimensional image is calculated. aver As shown in the following formula:

[0017] The S4 pulse wave signal denoising process described above involves the use of the Savitzky-Golay algorithm to smooth the waveform data after obtaining RGB channel data from a smartphone camera from the original PPG signal, due to the significant influence of high-frequency noise components. Using the Savitzky-Golay algorithm to smooth and denoise the data stream can improve the smoothness of the data stream, reduce the impact of noise on the data, and increase the signal-to-noise ratio.

[0018] Furthermore, the Savitzky-Golay algorithm employs the least squares principle and uses polynomial fitting for filtering. It takes N points to the left and right of the filtering point, for a total of 2N+1 points, for fitting. The steps for smoothing and denoising using the Savitzky-Golay algorithm are as follows: The Savitzky-Golay algorithm has three important parameters: nl, nr, and degree. This invention uses a symmetric filter, so nl = nr. The degree polynomial order is fitted using nl + nr + 1 points. At most one nl + nr + 1 points can determine an nl + nr polynomial, requiring nl + nr > degree. Calculate the coefficients of the polynomial matrix in the Savitzky-Golay algorithm; The dataset is smoothed by using the coefficient matrix of the Savitzky-Golay filter.

[0019] Furthermore, the smoothed pulse wave PPG signal includes limb tremors, breathing, mechanical vibrations of the instrument, and power frequency interference noise during the user's test. It is necessary to remove the interference of high and low frequency noise in the pulse wave signal. Since the energy of the pulse signal is mainly concentrated in the low frequency, this invention uses a Butterworth filter to filter out high and low frequency noise, setting the filter order to 3 and the bandpass range to 0.2-20Hz.

[0020] Furthermore, low-frequency interference caused by human respiration is the main cause of baseline drift in pulse wave signals. Although Butterworth filtering improves the baseline drift to some extent, it does not completely eliminate it. Cubic spline interpolation is used to completely remove baseline drift in pulse signals.

[0021] The specific steps of the cubic spline interpolation method are as follows: First, it is necessary to calculate the starting point of each single-cycle waveform in the pulse signal. By calculating the fundamental frequency of the pulse signal, the average length of the single-cycle pulse can be obtained. Based on this length, a window is designed and slides backward in the pulse signal. In each window, the valley value of the pulse signal can be obtained. When the amplitude of all points within a certain threshold range to the left and right of a certain point in the window is less than the amplitude of that point, it can be determined as a valley value, which is the starting point of the waveform. Therefore, during the window sliding process, the starting points of all single-cycle pulse waveforms in the pulse signal can be obtained. Based on the waveform starting points obtained above, cubic spline interpolation is used for fitting to obtain the fitted line. Finally, the pulse signal is subtracted from the fitted baseline to obtain the corrected signal.

[0022] The S5 average single-cycle pulse wave mentioned above contains multiple cycles in the pulse wave signal. The pulse wave signals of different cycles have slight differences due to changes in human body state and interference. Using the average single-cycle pulse wave to represent each sample data is more objective and accurate.

[0023] Furthermore, since abnormal periodic signals may exist during the pulse signal acquisition process due to external factors, such as wrist movement or shaking, in order to avoid abnormal signals affecting subsequent analysis, signals that are too long or too short are discarded based on the mean and variance of the length of all single-cycle pulse waves, so that the selected single-cycle pulse waves are representative. Then, the average value of each cycle pulse is calculated to obtain the average waveform of the pulse signal.

[0024] The S6 pulse signal thresholdless recursive graph is a nonlinear dynamic analysis method that converts the pulse wave signal into a thresholdless recursive graph in order to preserve more detailed features in the pulse signal.

[0025] Furthermore, the thresholdless recursive graph of the pulse signal uses the C-C algorithm to calculate the optimal phase space reconstruction parameters for each sample. Since the single-cycle length of different pulse signals is different, zero padding is performed at the end of the pulse signal with a shorter cycle to generate a thresholdless recursive graph of the same size. Then, the VGG-16 network is used to perform feature self-learning on the thresholdless recursive graph to establish a pulse signal classification model.

[0026] The S7 method establishes an integrated classification model for pulse signals. Commonly used pulse analysis methods include time-domain analysis, time-frequency domain analysis, and nonlinear dynamic analysis. Each of these three methods has its own advantages and disadvantages. In order to integrate the advantages of different pulse signal analysis methods, this invention proposes an integrated classification model for pulse signals based on the Stacking method.

[0027] Furthermore, based on the parameter types extracted by each analysis method, appropriate algorithms are selected to establish classification models. For structured feature parameters extracted by time-domain and time-frequency domain analysis methods, SVM is used to establish classification models. For nonlinear dynamic methods, thresholdless recursive graphs and ResNet networks are used to establish classification models. Finally, the classification models are integrated using the Stacking method. By combining the advantages of different analysis methods and classification models, the classification and recognition accuracy of pulse signals is further improved.

[0028] Furthermore, the specific steps for establishing the pulse signal integrated classification model are as follows: First, the pulse signal is preprocessed to extract the feature parameters in the time domain and time-frequency domain, and the pulse signal is converted into a thresholdless recursive graph to prepare data for the training of the base learner. Then, SVM and ResNet were selected as base learners to build classification models corresponding to the three analysis methods respectively, and the output results were concatenated to form a new dataset. Finally, the newly generated dataset is used to train the meta-learner, thereby establishing an integrated classification model for pulse signals.

[0029] Furthermore, SVM employs a specific nonlinear mapping relationship to project feature parameters into a high-dimensional space, enabling them to have better separability in the high-dimensional space. Based on the distribution of feature parameters in the high-dimensional space, a hyperplane that can distinguish different types is constructed through training, thereby obtaining classification results.

[0030] Furthermore, ResNet is a deep convolutional neural network model whose most significant feature is that it increases network depth by utilizing residuals, thereby extracting deeper features. The core idea of ​​ResNet is to represent the difference between the input mapping and the optimal solution as a function. By using residuals, it not only increases the number of network layers but also solves the network degradation problem.

[0031] Furthermore, this invention employs a fully connected neural network FCNN as the meta-learner of the ensemble model. Considering the simplicity and non-linearity of the meta-learner, a three-layer fully connected neural network is established. The first and second layers each contain 1024 neurons, and the third layer contains 512 neurons, all using the ReLU activation function. The output layer contains 7 neurons and uses the softmax activation function.

[0032] Furthermore, the aforementioned time-frequency domain analysis method employs wavelet packet analysis, which uses different wavelet bases to decompose the signal. This method has significant advantages in analyzing unstable signals. As a time-frequency analysis method, wavelet packet analysis can simultaneously zoom in on both time-domain and frequency-domain information, possessing excellent time-frequency local analysis capabilities. While decomposing the low-frequency band, wavelet packet analysis can also further decompose the high-frequency band, improving the time-frequency resolution of the signal. Since the energy of the pulse is mainly concentrated in the frequency band below 20Hz, the energy characteristics of the pulse signal in different frequency bands are extracted through eight-layer wavelet packet decomposition.

[0033] Furthermore, the wavelet packet analysis method employs wavelet analysis, which has good regularity and symmetry, reducing phase distortion caused during calculation and obtaining multidimensional energy characteristics.

[0034] The S8 pulse classification analysis results output extracts the time-domain and time-frequency domain features of the PPG pulse signal in the pulse signal analysis and converts them into a threshold-free recursive graph. Based on the eight-factor pulse analysis, a classification framework applicable to 28 pulse types in traditional Chinese medicine is proposed by combining the Stacking method with an integrated classification model. A classification model is established based on the existing dataset. Finally, the optimal data results are displayed to mobile terminal users.

[0035] Furthermore, the output of the S8 pulse classification analysis results includes storing the collected user information, pulse data, and analysis results, and provides a data query function.

[0036] Furthermore, the output of the S8 pulse classification analysis results, due to the large variety of pulse types, decomposes them into eight finger-feel sensations to reflect the basic characteristics of pulses. These eight finger-feel sensations correspond to eight factors: pulse position, pulse rate, pulse force, pulse width, pulse length, evenness, fluency, and tension. Each factor represents a pulse characteristic. From a mathematical set perspective, each pulse is composed of a non-empty subset of these eight factors. Each factor contains different pulse types. There are 28 common pulse types. By combining the eight factors of pulse with time-domain, time-frequency domain, and nonlinear dynamic analysis methods, effective feature parameters are extracted from multiple perspectives and combined with the Stacking method to establish a pulse signal classification model. By establishing the classification model, the classification results of the pulse in each factor can be obtained. At the same time, the output of the pulse classification analysis results can identify multiple combined pulses without establishing a classification model for all pulse types. Attached Figure Description

[0037] Figure 1 Flowchart for collecting mobile phone pulse wave health data using mobile phone camera Figure 2 Schematic diagram of PPG signal extraction by mobile phone camera Figure 3 Flowchart for establishing an integrated classification model of pulse signals Figure 4 Eight factors of pulse diagnosis and 28 types of pulse patterns in Traditional Chinese Medicine Specific implementation methods

[0038] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. In addition, the forms of the various structures described in the following embodiments are merely illustrative. The method for detecting the impact of obesity on the health of internal organs based on mobile terminal PPG signals involved in the present invention is not limited to the following embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Detailed Implementation

[0039] like Figure 1 As shown, this invention provides a method for collecting finger pulse health data based on a mobile terminal's mobile phone camera, comprising the following steps: S1 captures video. S2 photoplethysmography (PPG) signal preprocessing S3 extracts the raw PPG signal from the photoplethysmography (PPG) wave. S4 pulse wave signal denoising processing S5 average single-cycle pulse wave S6 pulse signal threshold-free recursive plot S7 establishes an integrated classification model for pulse signals. Output of S8 pulse classification analysis results.

[0040] The video acquisition method described in S1 uses a mobile phone camera to capture the photoplethysmography (PPG) signal of the index fingertip. The sampling frequency is 20-200Hz, and the video is recorded at a speed of 20-60 frames per second for a duration of 20-120 seconds.

[0041] The S2 photoplethysmography (PPG) signal preprocessing module needs to convert the camera's YUV format data every 50 milliseconds, convert the numerical matrix into PPG signal values ​​according to the PPG signal value calculation algorithm, and perform smoothing and noise reduction processing on the continuously generated PPG datasets.

[0042] The YUV format conversion mentioned above involves calling the onPreviewFrame method in Android during Camera preview. The variable data stores the YUV format byte matrix of the current video frame. During processing, this variable data needs to be converted from YUV format to RGB format.

[0043] Furthermore, there is a fixed conversion formula for the mutual conversion between YUV and RGB formats. However, the coefficients in the formula are all in non-integer form, which involves a large number of floating-point operations in the programming implementation, which will seriously affect the performance of the conversion algorithm. This invention provides an efficient YUV conversion method that transforms the performance-intensive floating-point operations into extremely fast bitwise operations, thereby improving the performance of the conversion algorithm.

[0044] Furthermore, the improved algorithm formula for YUV to RGB conversion is as follows:

[0045] The simplified formula is as follows:

[0046] The formula, after further optimization, is as follows:

[0047] By replacing all multiplication operations with bit shifting operations, the above formula can be easily optimized using bit shifting, as shown below:

[0048] In computer systems, addition, subtraction, and shifting algorithms are far more efficient than multiplication. Therefore, multiplication needs to be completely replaced, and the algorithm needs further optimization, as shown in the following formula:

[0049] The S3 process extracts the raw PPG signal. In the RGB color model, the fluctuation trend of the red channel most closely matches the fluctuation trend of the PPG signal value curve; therefore, the red channel value X from the video stream data is selected. r As a calculation variable, the average red value R of each pixel in the two-dimensional image is calculated. aver As shown in the following formula:

[0050] The S4 pulse wave signal denoising process described above involves the use of the Savitzky-Golay algorithm to smooth the waveform data after obtaining RGB channel data from a smartphone camera from the original PPG signal, due to the significant influence of high-frequency noise components. Using the Savitzky-Golay algorithm to smooth and denoise the data stream can improve the smoothness of the data stream, reduce the impact of noise on the data, and increase the signal-to-noise ratio.

[0051] Furthermore, the Savitzky-Golay algorithm employs the least squares principle and uses polynomial fitting for filtering. It takes N points to the left and right of the filtering point, for a total of 2N+1 points, for fitting. The steps for smoothing and denoising using the Savitzky-Golay algorithm are as follows: The Savitzky-Golay algorithm has three important parameters: nl, nr, and degree. This invention uses a symmetric filter, so nl = nr. The degree polynomial order is fitted using nl + nr + 1 points. At most one nl + nr + 1 points can determine an nl + nr polynomial, requiring nl + nr > degree.

[0052] Calculate the coefficients of the polynomial matrix in the Savitzky-Golay algorithm.

[0053] The dataset is smoothed by using the coefficient matrix of the Savitzky-Golay filter.

[0054] Furthermore, the smoothed pulse wave PPG signal includes limb tremors, breathing, mechanical vibrations of the instrument, and power frequency interference noise during the user's test. It is necessary to remove the interference of high and low frequency noise in the pulse wave signal. Since the energy of the pulse signal is mainly concentrated in the low frequency, this invention uses a Butterworth filter to filter out high and low frequency noise, setting the filter order to 3 and the bandpass range to 0.2-20Hz.

[0055] Furthermore, low-frequency interference caused by human respiration is the main reason for baseline drift in pulse wave signals. Although Butterworth filtering improves the baseline drift to some extent, it does not completely eliminate it. Cubic spline interpolation is used to completely remove baseline drift from the pulse wave signal, resulting in a PPG pulse wave signal as follows: Figure 2 As shown.

[0056] The specific steps of the cubic spline interpolation method are as follows: First, the starting point of each single-cycle waveform in the pulse signal needs to be calculated. By calculating the fundamental frequency of the pulse signal, the average length of a single-cycle pulse can be obtained. Based on this length, a window is designed and slid backward in the pulse signal. In each window, the valley value of the pulse signal can be obtained. When the amplitude of all points within a certain threshold range to the left and right of a certain point in the window is less than the amplitude of that point, it can be determined as a valley value, which is the starting point of the waveform. Therefore, during the window sliding process, the starting points of all single-cycle pulse waveforms in the pulse signal can be obtained. Based on the waveform starting points obtained above, cubic spline interpolation is used for fitting to obtain a fitted line. Finally, the pulse signal is subtracted from the fitted baseline to obtain the corrected signal. The obtained PPG pulse wave signal is as follows: Figure 2 As shown.

[0057] The S5 average single-cycle pulse wave is used because the pulse wave signal contains multiple cycles. The pulse wave signals of different cycles have slight differences due to changes in human body state and interference. Using the average single-cycle pulse wave to represent each sample data is more objective and accurate.

[0058] Furthermore, since abnormal periodic signals may exist during the pulse signal acquisition process due to external factors, such as wrist movement or shaking, in order to avoid abnormal signals affecting subsequent analysis, signals that are too long or too short are discarded based on the mean and variance of the length of all single-cycle pulse waves, so that the selected single-cycle pulse waves are representative. Then, the average value of each cycle pulse is calculated to obtain the average waveform of the pulse signal.

[0059] The S6 pulse signal thresholdless recursive graph is a nonlinear dynamic analysis method that converts the pulse wave signal into a thresholdless recursive graph in order to preserve more detailed features in the pulse signal.

[0060] Furthermore, the thresholdless recursive graph of the pulse signal uses the C-C algorithm to calculate the optimal phase space reconstruction parameters for each sample. Since the single-cycle length of different pulse signals is different, zero padding is performed at the end of the pulse signal with a shorter cycle to generate a thresholdless recursive graph of the same size. Then, the VGG-16 network is used to perform feature self-learning on the thresholdless recursive graph to establish a pulse signal classification model.

[0061] The S7 method establishes an integrated classification model for pulse signals. Commonly used pulse analysis methods include time-domain analysis, time-frequency domain analysis, and nonlinear dynamic analysis. Each of these three methods has its own advantages and disadvantages. In order to integrate the advantages of different pulse signal analysis methods, this invention proposes an integrated classification model for pulse signals based on the Stacking method.

[0062] Furthermore, based on the parameter types extracted by each analysis method, appropriate algorithms are selected to establish classification models. For structured feature parameters extracted by time-domain and time-frequency domain analysis methods, SVM is used to establish classification models. For nonlinear dynamic methods, thresholdless recursive graphs and ResNet networks are used to establish classification models. Finally, the classification models are integrated using the Stacking method. By combining the advantages of different analysis methods and classification models, the classification and recognition accuracy of pulse signals is further improved.

[0063] Furthermore, the establishment of an integrated classification model for pulse signals, such as... Figure 3 As shown, the specific steps are as follows: First, the pulse signal is preprocessed to extract the feature parameters in the time domain and time-frequency domain, and the pulse signal is converted into a thresholdless recursive graph to prepare data for the training of the base learner. Then, SVM and ResNet were selected as base learners to build classification models corresponding to the three analysis methods respectively, and the output results were concatenated to form a new dataset. Finally, the newly generated dataset is used to train the meta-learner, thereby establishing an integrated classification model for pulse signals.

[0064] Furthermore, SVM employs a specific nonlinear mapping relationship to project feature parameters into a high-dimensional space, enabling them to have better separability in the high-dimensional space. Based on the distribution of feature parameters in the high-dimensional space, a hyperplane that can distinguish different types is constructed through training, thereby obtaining classification results.

[0065] Furthermore, ResNet is a deep convolutional neural network model whose most significant feature is that it increases network depth by utilizing residuals, thereby extracting deeper features. The core idea of ​​ResNet is to represent the difference between the input mapping and the optimal solution as a function. By using residuals, it not only increases the number of network layers but also solves the network degradation problem.

[0066] Furthermore, this invention employs a fully connected neural network FCNN as the meta-learner of the ensemble model. Considering the simplicity and non-linearity of the meta-learner, a three-layer fully connected neural network is established. The first and second layers each contain 1024 neurons, and the third layer contains 512 neurons, all using the ReLU activation function. The output layer contains 7 neurons and uses the softmax activation function.

[0067] Furthermore, the aforementioned time-frequency domain analysis method employs wavelet packet analysis, which uses different wavelet bases to decompose the signal. This method has significant advantages in analyzing unstable signals. As a time-frequency analysis method, wavelet packet analysis can simultaneously zoom in on both time-domain and frequency-domain information, possessing excellent time-frequency local analysis capabilities. While decomposing the low-frequency band, wavelet packet analysis can also further decompose the high-frequency band, improving the time-frequency resolution of the signal. Since the energy of the pulse is mainly concentrated in the frequency band below 20Hz, the energy characteristics of the pulse signal in different frequency bands are extracted through eight-layer wavelet packet decomposition.

[0068] Furthermore, the wavelet packet analysis method described above has good regularity and symmetry, which can reduce phase distortion caused during calculation and obtain multidimensional energy characteristics.

[0069] The output of the S8 pulse classification analysis results involves extracting the time and frequency domain features of the PPG pulse signal in the pulse signal analysis and converting them into a threshold-free recursive graph. Based on the eight-factor analysis of pulse, a classification framework applicable to 28 types of pulse in traditional Chinese medicine is proposed by combining the Stacking method with an integrated classification model. A classification model is established based on the existing dataset. Finally, the optimal data results are displayed to mobile terminal users.

[0070] Furthermore, the output of the S8 pulse classification analysis results includes storing the collected user information, pulse data, and analysis results, and provides a data query function.

[0071] Furthermore, the output of the S8 pulse classification analysis, due to the large variety of pulse types, decomposes them into eight finger-based sensations to reflect their basic characteristics. These eight sensations correspond to eight factors: pulse location, pulse rate, pulse strength, pulse width, pulse length, evenness, fluency, and tension. Each factor represents a pulse characteristic. From a mathematical set theory perspective, each pulse is composed of a non-empty subset of these eight factors, and each factor contains different pulse types. The 28 common pulse types are as follows: Figure 4 As shown, by combining eight factors of pulse diagnosis with time-domain, time-frequency domain and nonlinear dynamic analysis methods, effective feature parameters are extracted from multiple perspectives and combined with the Stacking method to establish a pulse signal classification model. By establishing the classification model, the classification results of pulse diagnosis in each factor can be obtained. At the same time, the output of the pulse diagnosis analysis results can identify multiple combined pulses without establishing a classification model for all pulse types.

[0072] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for collecting finger pulse health data based on a mobile terminal's mobile phone camera, characterized in that, The method includes the following steps: S1 captures video. S2 photoplethysmography (PPG) signal preprocessing S3 extracts the raw PPG signal from the photoplethysmography (PPG) wave. S4 pulse wave signal denoising processing S5 average single-cycle pulse wave S6 pulse signal threshold-free recursive plot S7 establishes an integrated classification model for pulse signals. Output of S8 pulse classification analysis results.

2. The video acquisition method of S1 according to claim 1, characterized in that, The video acquisition method utilizes a mobile phone camera to capture the photoplethysmography (PPG) signal of the index fingertip, with a sampling frequency of 20–200 Hz, recording the video at a speed of 20–60 frames per second, and a video acquisition time of 20–120 seconds.

3. The S2 photoplethysmography (PPG) signal preprocessing according to claim 1, characterized in that, The photoplethysmography (PPG) signal preprocessing involves converting the camera's YUV format data every 50 milliseconds, converting the numerical matrix into PPG signal values ​​according to the PPG signal value calculation algorithm, and performing smoothing and noise reduction on the continuously generated PPG datasets. The YUV format conversion, in the Camera preview, calls the Android's onPreviewFrame method. The variable `data` stores the YUV format byte matrix of the current video frame, and during processing, this variable `data` needs to be converted from YUV format to RGB format.

4. The method for extracting the raw PPG signal of photoplethysmography (PPG) wave using S3 according to claim 1, characterized in that, The fluctuation trend of the red channel in the RGB color model of the extracted photoplethysmography (PPG) raw signal is closest to the fluctuation trend of the PPG signal value curve. Therefore, the red channel value X in the video stream data is selected. r As a calculation variable, the average red value R of each pixel in the two-dimensional image is calculated. aver。 5. The S4 pulse wave signal denoising processing according to claim 1, characterized in that, In the pulse wave signal denoising process, the high-frequency components of noise have a significant impact. After obtaining the RGB channel data from the smartphone camera, the Savitzky-Golay algorithm is used to smooth the waveform data. Using the Savitzky-Golay algorithm to smooth and denoise the data stream can improve the smoothness of the data stream, reduce the impact of noise on the data, and increase the signal-to-noise ratio. The Savitzky-Golay algorithm adopts the least squares principle and uses a polynomial fitting method for filtering, taking N points to the left and right of the filtering point, for a total of 2N+1 points for fitting. The smoothed pulse wave PPG signal includes limb tremors, breathing, mechanical vibration of the instrument, and power frequency interference noise during the user's test. It is necessary to remove the interference of high and low frequency noise in the pulse wave signal. Since the energy of the pulse signal is mainly concentrated in the low frequency, the Butterworth filter is used to filter out high and low frequency noise. The filter order is set to 3, and the bandpass range is 0.2-20Hz.

6. The S4 pulse wave signal denoising processing according to claim 1, characterized in that, In the pulse wave signal denoising process, low-frequency interference caused by human respiration is the main cause of baseline drift in the pulse wave signal. Although the baseline drift phenomenon is improved to some extent after Butterworth filtering, it is not completely eliminated. The baseline drift in the pulse signal can be completely removed by using cubic spline interpolation.

7. The S5 average single-cycle pulse wave according to claim 1, characterized in that, The average single-cycle pulse wave processing is used because the pulse wave signal contains multiple cycles. The pulse wave signals of different cycles have slight differences due to changes in human body state and interference. Using the average single-cycle pulse wave to represent each sample data is more objective and accurate. Based on the mean and variance of the length of all single-cycle pulse waves, signals that are too long or too short are discarded to make the selected single-cycle pulse waves representative. Then, the average value of the pulse wave in each cycle is calculated to obtain the average waveform of the pulse wave signal.

8. The S6 pulse signal thresholdless recursive graph according to claim 1, characterized in that, The thresholdless recursive graph of pulse signals is a nonlinear dynamic analysis method that converts pulse wave signals into thresholdless recursive graphs to preserve more detailed features in the pulse signals. The thresholdless recursive graph of pulse signals uses the C-C algorithm to calculate the optimal phase space reconstruction parameters for each sample. Since different pulse signals have different single-cycle lengths, zeros are padded at the end of pulse signals with shorter cycles to generate thresholdless recursive graphs of the same size. Then, the VGG-16 network is used to perform feature self-learning on the thresholdless recursive graph to establish a pulse signal classification model.

9. The pulse signal integrated classification model established in S7 according to claim 1, characterized in that, The establishment of an integrated classification model for pulse signals is based on the fact that commonly used pulse analysis methods include time-domain analysis, time-frequency domain analysis, and nonlinear dynamic analysis. Each of these methods has its own advantages and disadvantages. To integrate the strengths of different pulse signal analysis methods, an integrated classification model for pulse signals based on the Stacking method is proposed. Appropriate algorithms are selected to build classification models according to the parameter types extracted by each analysis method. For structured feature parameters extracted by time-domain and time-frequency domain analysis methods, SVM is used to build the classification model. For nonlinear dynamic methods, thresholdless recursive graphs and ResNet networks are used to build the classification model. Finally, the classification model is integrated using the Stacking method, combining the advantages of different analysis methods and classification models to further improve the accuracy of pulse signal classification and recognition.

10. The output of the S8 pulse classification analysis result according to claim 1, characterized in that, To obtain better output results from the pulse classification analysis and to more objectively reflect the basic characteristics of the pulse, the pulse classification is decomposed into eight types of finger sensations, corresponding to eight factors: pulse position, pulse rate, pulse strength, pulse width, pulse length, evenness, fluency, and tension. Each factor represents a pulse characteristic. From a mathematical set perspective, each pulse is composed of a non-empty subset of these eight factors, and each factor contains different pulse types. The 28 common TCM pulse types are analyzed using the eight factors combined with time-domain, time-frequency domain, and nonlinear dynamics analysis methods. Effective feature parameters are extracted from multiple perspectives and combined with the Stacking method to establish the pulse signal classification model. The established classification model can obtain the classification results of the pulse in each factor. At the same time, the output of the pulse classification analysis results can identify multiple combined pulses without establishing classification models for all pulse types.

Images