Non-contact traditional Chinese medicine exterior and interior syndrome identification method and system based on HRV data

Abstract

The invention provides a non-contact traditional Chinese medicine exterior and interior syndrome identification method and system based on HRV data, and relates to the technical field of biological detection.The method comprises the steps that non-contact face video collection is conducted on an object to be detected through remote photoplethysmography, and an original face video is collected; preprocessing each frame of face image in the original face video to obtain an ROI pixel mean value of the face image, and forming a channel pixel change curve according to the ROI pixel mean value; performing source signal separation on the channel pixel change curve to obtain a plurality of source signals, and selecting the source signal with the highest Pearson correlation coefficient in the green channel as an rPPG signal; performing spline interpolation on the rPPG signal, performing peak detection to obtain an RR interval sequence, and calculating an HRV feature set according to the RR interval sequence; and constructing a syndrome identification model, and discriminating the HRV feature set of the to-be-detected object to obtain a traditional Chinese medicine exterior and interior syndrome identification result of the to-be-detected object. According to the invention, the perception capability of syndrome identification on the slight difference of the physiological state is improved.

Description

Technical Field

[0001] This invention relates to the field of biological detection technology, and in particular to a non-contact method and system for identifying the exterior and interior syndromes in Traditional Chinese Medicine based on HRV data. Background Technology

[0002] Heart rate variability (HRV), as an important indicator reflecting the regulation of cardiovascular activity by the autonomic nervous system, can provide insights into the functional activity of the human autonomic nervous system and reveal the characteristics of autonomic nervous function in different diseases and states. Existing research has fully demonstrated a close correlation between HRV and Traditional Chinese Medicine (TCM) syndromes. That is, different HRV indicators can reveal the intrinsic connection between autonomic nervous function and exterior-interior syndromes from different perspectives. Traditional TCM syndrome diagnosis methods struggle to monitor these changes in real time. However, HRV indicators can reflect changes in autonomic nervous function in real time, and by periodically monitoring HRV indicators, changes in the patient's TCM exterior-interior syndromes can be dynamically monitored. This provides a foundation for TCM syndrome differentiation and treatment.

[0003] Chinese Patent CN116109818B discloses a TCM pulse diagnosis system, method, and apparatus based on facial video. The method includes: S1. A video acquisition module acquires the user's original facial video data; S2. A ROI detection unit extracts the region of interest (ROI) from each frame of the original facial video data to obtain a calculable facial region; S3. A signal extraction unit extracts the green spectrum of the calculable facial region and obtains a temporal feature signal X by weighted summation of the output light equalization weights; S4. A filtering unit filters the temporal feature signal X to obtain a pulse waveform signal; S5. A pulse diagnosis module trains the pulse waveform signal using a neural network to determine the pulse type. However, the above solution only analyzes macroscopic pulse waveforms, and its perception capability is limited to waveform morphological changes caused by significant changes in heart rate or blood volume, easily leading to insufficient perception of physiological states. Therefore, it is essential to provide a non-contact TCM syndrome differentiation method and system based on HRV data to improve the ability of syndrome differentiation to perceive subtle differences in physiological states. Summary of the Invention

[0004] In view of this, the present invention proposes a non-contact TCM syndrome differentiation method and system based on HRV data.

[0005] This invention provides a non-contact TCM syndrome differentiation method based on HRV data, the method comprising: Remote optical volumetric imaging is used to perform non-contact facial video acquisition of the test object, capturing the original facial video. Each frame of the facial image in the original facial video is preprocessed to obtain the average ROI pixel value corresponding to the RGB three channels in the facial image, and a channel pixel change curve is formed based on the average ROI pixel value. Source signal separation is performed on the channel pixel change curve to obtain multiple source signals corresponding to the channel pixel change curve, and the source signal with the highest Pearson correlation coefficient in the green channel is selected as the rPPG signal; After spline interpolation of the rPPG signal, peak detection is performed to obtain the RR interval sequence, and the HRV feature set is calculated based on the RR interval sequence, wherein the HRV feature set includes time domain features, frequency domain features and nonlinear feature parameters; Based on the HRV feature set and the TCM exterior-interior syndrome labels, a syndrome identification model is constructed. The HRV feature set of the test subject is input into the syndrome identification model for discrimination to obtain the TCM exterior-interior syndrome identification results of the test subject.

[0006] Based on the above technical solutions, preferably, the preprocessing of each frame of facial image in the original facial video specifically includes: Face detection and facial key point localization are performed on each frame of the original facial video to obtain the face bounding box and facial key points, wherein the face bounding box includes multiple facial anatomical regions; Based on the facial key points, the region of interest corresponding to the facial anatomical region in the face bounding box is determined, and the non-skin region in the region of interest is removed by foreground segmentation to obtain the standard region of interest. Affine registration is performed on the standard regions of interest in adjacent frame facial images, and the standard regions of interest in the current frame facial image are aligned to the reference frame. In each frame of the facial image, the pixel intensity statistics of the RGB three channels are calculated separately within the standard region of interest to obtain the average pixel value of the RGB three channels in the facial image.

[0007] Based on the above technical solution, preferably, the step of separating the source signals of the channel pixel change curve to obtain multiple source signals corresponding to the channel pixel change curve specifically includes: An observation matrix is ​​constructed in the time dimension based on the channel pixel change curve, and the observation matrix is ​​whitened to obtain a whitening matrix; Blind source separation is performed on the whitening matrix to obtain multiple source signal components. Based on physiological priors and quality indicators, source signal components related to heartbeat are selected from all source signal components, labeled, and used as candidate source signals. The remaining unlabeled source signal components are labeled as respiratory, light change, motion artifacts, or noise components according to their spectrum and correlation. The candidate source signals are reconstructed in time series based on the separation matrix to obtain multiple source signals corresponding to the channel pixel change curve.

[0008] More preferably, the blind source separation includes any one of independent component analysis, nonnegative matrix factorization, principal component analysis combined with independent rotation, and sparse component analysis.

[0009] More preferably, the step of performing peak detection after spline interpolation of the rPPG signal to obtain the RR interval sequence specifically includes: The rPPG signal is resampled along the time axis at a preset sampling rate to obtain a uniform sampling sequence. Local maxima detection is performed on the filtered signal obtained after filtering the uniform sampling sequence to obtain a set of candidate peaks; Parabolic fitting is performed in the neighborhood of each candidate peak in the candidate peak set to obtain the candidate peak position, and the RR interval sequence is calculated based on the time difference between any two adjacent selected peak positions.

[0010] More preferably, the step of inputting the HRV feature set of the test object into the syndrome identification model for discrimination specifically includes: Obtain the HRV feature set of the object to be tested, and align the HRV feature set with the same order and the same name fields according to the training stage to obtain preprocessed features; The preprocessed features are input into the syndrome identification model to obtain the probability distribution of each syndrome category. Based on the preset discrimination threshold, the probability distribution of each syndrome category is mapped to the final syndrome category. In the case of multi-classification, the category corresponding to the largest posterior probability is selected to output the TCM exterior-interior syndrome identification result.

[0011] More preferably, the facial anatomical region is a skin region including at least one of the forehead region and the cheekbone region, and the skin region is determined within the face bounding box based on facial key points; Alternatively, the facial anatomical region may be the forehead region located between the brow bone and the hairline, and / or the cheekbone region located between the lower edge of the eye socket, the outer side of the nasal wing, and the cheekbone arch.

[0012] A second aspect of this application provides a non-contact TCM syndrome differentiation system based on HRV data. The non-contact TCM syndrome differentiation system includes a data acquisition module, a data processing module, and a syndrome identification module, wherein... The data acquisition module is used to perform non-contact facial video acquisition of the test object through remote optical volumetric tracing, and to acquire the original facial video. The data processing module is used to preprocess each frame of facial image in the original facial video, obtain the average ROI pixel value corresponding to the RGB three channels in the facial image, form a channel pixel change curve based on the average ROI pixel value, perform source signal separation on the channel pixel change curve to obtain multiple source signals corresponding to the channel pixel change curve, select the source signal with the highest Pearson correlation coefficient in the green channel as the rPPG signal, perform spline interpolation on the rPPG signal and perform peak detection to obtain the RR interval sequence, and calculate the HRV feature set based on the RR interval sequence, wherein the HRV feature set includes time domain features, frequency domain features and nonlinear feature parameters; The syndrome identification module is used to construct a syndrome identification model based on the HRV feature set and the TCM exterior-interior syndrome labels. The HRV feature set of the test subject is input into the syndrome identification model for discrimination to obtain the TCM exterior-interior syndrome identification result of the test subject.

[0013] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory.

[0014] In a fourth aspect, this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of a non-contact TCM syndrome differentiation method based on HRV data.

[0015] The non-contact TCM syndrome differentiation method and system based on HRV data provided by this invention has the following advantages over existing technologies: (1) In the preprocessing stage, the facial ROI is extracted and the mean values ​​of the RGB three channels are calculated to construct the channel pixel change curve, which can effectively suppress random noise and local texture interference. By separating the source signal, the physiological components are separated from the illumination changes, head movements and camera noise, which improves the signal-to-noise ratio of the pulse-related components. Furthermore, the source with the highest Pearson correlation coefficient with the green channel is used as the rPPG signal. Taking advantage of the physical characteristic that the green channel is more sensitive to changes in blood volume, the accuracy and robustness of cardiac information extraction are further improved. After spline interpolation, peak detection is performed to smooth the discrete error caused by irregular sampling. This study aims to improve the stability of cardiac rhythm localization and RR interval estimation, reduce the false negative rate, and systematically extract time-domain, frequency-domain, and nonlinear features from the RR interval sequences reconstructed by rPPG to form a feature set covering the multidimensional mechanisms of autonomic nervous system regulation. This feature set can more sensitively characterize changes in sympathetic / parasympathetic balance, stress load, rhythmicity, and complexity, thereby improving the ability of the syndrome identification model to perceive subtle differences in physiological states. Furthermore, by supervising the learning of HRV features and TCM syndrome labels, a mapping relationship from physiological signals to syndrome representations can be established, reducing the subjectivity and consistency problems caused by relying on purely empirical judgments and improving the stability of diagnosis.

[0016] (2) By obtaining bounding boxes and anatomical key areas through face detection and facial key point localization, signal extraction is limited to areas with more stable blood flow information and less facial expression interference, reducing noise introduced by non-target pixels. Skin and non-skin separation is performed on the region of interest, and non-skin pixels are removed, significantly reducing texture and reflection noise unrelated to blood volume changes. Affine registration is performed on the standard ROI of adjacent frames, thereby improving the reliability of subsequent peak detection and RR interval estimation, reducing false detection or missed detection. At the same time, pixel intensity statistics are calculated for the R, G and B channels in the standard ROI respectively, and random noise of a large number of pixels is averaged and suppressed, enhancing physiological cycle components. The G channel is more sensitive to changes in blood oxygen and blood volume, and the channel-level statistics can highlight weak cardiac-related signals, providing higher quality input for subsequent source separation and correlation screening. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a non-contact TCM syndrome differentiation method based on HRV data provided by this invention; Figure 2 This is a schematic diagram of the non-contact TCM syndrome differentiation system provided by the present invention. Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0019] Explanation of reference numerals in the attached diagram: 1. Non-contact TCM syndrome differentiation system; 11. Data acquisition module; 12. Data processing module; 13. Syndrome differentiation module; 2. Electronic device; 21. Processor; 22. Communication bus; 23. User interface; 24. Network interface; 25. Memory. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention discloses a non-contact TCM syndrome differentiation method based on HRV data, with reference to... Figure 1 The steps of this method include S1 to S5.

[0022] Step S1: Non-contact facial video acquisition of the test object is performed by remote optical volumetric tracing to acquire the original facial video.

[0023] In this step, an image acquisition device with visible light imaging capability is selected, and the frame rate, resolution, and exposure parameters are set to meet the detection requirements of weak pulse-related signals. Under stable ambient light or uniform illumination by auxiliary continuous light sources, the camera and the subject are arranged according to a predetermined geometric relationship, and the background, distance, and angle are controlled. The subject is guided to keep their face directly facing the camera and minimize head movement and facial expression changes, and the acquisition time is recorded. Video acquisition is started and frame timestamps and camera intrinsic parameters are recorded to obtain a continuous raw facial video sequence. The signal quality index is detected during or after acquisition, and a prompt or re-acquisition is triggered when the quality index is lower than the threshold.

[0024] Specifically, a regular webcam is placed 0.5m directly in front of the person's face, or it can be mounted above a computer monitor. The webcam is connected to the computer via USB. The operator uses Matlab software to control the webcam to capture video, ensuring that the person's face is completely within the video frame. All videos are continuously captured for 10 seconds at a resolution of 640×480, a frame rate of 30 frames per second, and RGB color space, and saved in AVI format.

[0025] Step S2: Preprocess each frame of the facial image in the original facial video to obtain the average ROI pixel value corresponding to the RGB three channels in the facial image, and form a channel pixel change curve based on the average ROI pixel value.

[0026] The original RGB video frame images are extracted from the captured facial video. Face detection is performed on each RGB image frame. To reduce background interference, redundant background images are removed to obtain the face RGB image. Based on the distribution of blood vessels and blood flow in the face, regions of interest (ROIs) with dense blood vessel distribution and sufficient blood supply are selected, and R, G, and B single-channel images are obtained. By calculating the pixel mean of each ROI image in the three single channels, curves are plotted to reflect the changes in facial lighting caused by heartbeat, thus obtaining three facial pixel change curves.

[0027] This step also includes steps S21 to S24.

[0028] Step S21: Perform face detection and facial key point localization on each frame of the original facial video to obtain the face bounding box and facial key points, wherein the face bounding box includes multiple facial anatomical regions.

[0029] The facial anatomical region is a skin region including at least one of the forehead region and the zygomatic region, the skin region being defined within the face bounding box based on facial key points; or the facial anatomical region is the forehead region located between the brow ridge and the hairline and / or the zygomatic region located between the lower edge of the eye socket, the outer side of the nasal ala and the zygomatic arch.

[0030] In this step, a face detector is run on each frame of facial image, outputting at least one face bounding box and its confidence score. When multiple faces are detected, the face bounding box with the highest confidence score and the largest area is retained. Facial key point localization is performed within the face bounding box to obtain a set of key points such as facial contours, eyebrows, eyes, nose, and mouth. Based on the geometric relationship of the key points, the face bounding box is divided into multiple facial anatomical regions, which include at least the forehead, cheekbones, bridge / alae of the nose, and upper lip / jaw. Temporal tracking and smoothing are performed on the face bounding box and key points. If a single frame detection fails or the confidence score is lower than the threshold, the state of the previous frame is extrapolated and corrected in the next valid frame. When face occlusion, deflection, or blurring causes the key point reprojection error to exceed the threshold, the frame is marked as low confidence, and subsequent steps can choose to skip or reduce the weight.

[0031] Step S22: Based on multiple facial anatomical regions in the face bounding box, determine the region of interest corresponding to the facial anatomical region, and remove the background of non-skin regions in the region of interest through foreground segmentation to obtain the standard region of interest.

[0032] In this step, based on the geometric relationships of facial key points, polygons / masks are constructed within the face bounding box for multiple facial anatomical regions such as the forehead, cheekbones, nose, perioral region, and jaw, resulting in the original set of regions of interest ROIraw={R k}; Scale and shrink the original regions of interest (ROIs) according to the face size, restricting the regions to be within the bounding box and avoiding hair, eye sockets, and lip boundaries, resulting in normalized ROIs; Perform skin foreground segmentation within ROIs, removing non-skin pixels and background interference, to obtain standard ROIs = {S k};For S k Temporal smoothing and morphological filling are performed to ensure regional connectivity and cross-frame stability, and to provide a stable mask for subsequent registration.

[0033] Step S23: Perform affine registration on the standard regions of interest in the facial images of adjacent frames, and align the standard regions of interest in the facial image of the current frame to the reference frame.

[0034] In this step, a reference frame F is selected at the beginning of the video or at the moment of optimal quality. ref and facial key points and standard region of interest mask S ref When the quality of the reference frame degrades, an update is triggered. Feature points or grid point sets for registration are extracted from the standard regions of interest (ROIs) of the reference frame and the current frame. Based on the corresponding point pairs of the reference frame and the current frame, a robust estimation method is used to solve the two-dimensional affine transformation T (including scaling, rotation, translation, and shearing). The standard ROI mask and corresponding image blocks of the current frame are resampled and aligned to the coordinate system of the reference frame using T. The registration quality index is calculated. If it is lower than the threshold, the process reverts to the last valid transformation or triggers re-estimation and reference frame update.

[0035] In one example, the initial reference frame is selected as F from the first N frames (e.g., N=30) as the frame with the highest quality score Q. ref When the quality of M consecutive frames (e.g., M=15) is significantly better than the current reference frame, and the registration residual with the reference frame remains consistently high, the reference frame is switched and the accumulated drift is reset. Among the keypoints output by S21, points geometrically related to the ROI are selected as control points (e.g., outer corner of the eye, nasal wing, and corner of the mouth for the cheekbone region; several points above the eyebrow for the forehead region). Several auxiliary grid points are then evenly distributed within the ROI and tracked between adjacent frames using optical flow (PyrLK) to obtain corresponding point pairs. Alternatively, features are extracted within the ROI, and the reference frame is matched with the current frame. A ratio check and bidirectional consistency filtering are used to obtain reliable matching pairs.

[0036] Step S24: Calculate the pixel intensity statistics for the RGB three channels in the standard region of interest in each frame of the facial image to obtain the average pixel value of the RGB three channels in the facial image.

[0037] In this step, in the current frame, a standard region of interest (ROI) mask is applied to each facial anatomical region to extract the effective pixel set within the mask; pixels with high brightness, saturation, shadows, and unstable edges are removed or downweighted, and pixels / regions are weighted according to registration quality and skin confidence; the mean or robust mean (truncated mean / weighted median) is calculated for each region and each channel, and the statistics of each region are aggregated by area or confidence to obtain the R, G, and B channel mean values ​​of the full-face ROI; the channel mean values ​​of each frame are used to form a channel pixel change curve according to the timestamp, and the frame-level quality index and effective pixel ratio are output.

[0038] In this embodiment, face detection and facial key point localization obtain bounding boxes and anatomical key regions (forehead, cheekbones, sides of the nose, jaw, etc.), limiting signal extraction to areas with more stable blood flow information and less facial expression interference, reducing noise introduced by background / non-target pixels. The multi-anatomical region design allows for subsequent region weighting or robust fusion, mitigating the impact of local occlusion, oiliness, and specular reflection on a single region. Skin / non-skin separation is performed on the region of interest (ROI), removing non-skin pixels such as hair, eyebrows, beard, eyes, lips, teeth, and jewelry, significantly reducing texture and reflection noise unrelated to blood volume changes. Affine registration is performed on standard ROIs in adjacent frames, aligning the current frame to the reference frame, significantly reducing pixel mixing and brightness drift caused by head micro-movements, facial expression changes, and camera shake. The changes in time-series pixel statistics after alignment are mainly caused by blood flow rather than geometric pose changes, thereby improving the reliability of subsequent peak detection and RR interval estimation, and reducing false positives or false negatives. Within the standard ROI, pixel intensity statistics are calculated for the R, G, and B channels respectively. This averages and suppresses random noise from a large number of pixels, enhancing the physiological cycle component. The G channel is more sensitive to changes in blood oxygenation and blood volume; channel-level statistics can highlight weak cardiac-related signals, providing higher-quality input for subsequent source separation and correlation screening. The combination of anatomical region division, skin segmentation, and registration can, to some extent, mitigate intensity shifts caused by uneven lighting, specular reflection, and differences in skin color / makeup. Improved consistency of channel statistics in the temporal domain helps achieve better results in subsequent detrending and normalization steps. Standard ROI definition and alignment ensure that channel curves obtained from different individuals, devices, and scenarios have consistent data distribution and spectral characteristics, reducing inter-domain shifts and improving the generalization ability and stability of the trained model in actual deployment.

[0039] Step S3: Perform source signal separation on the channel pixel change curve to obtain multiple source signals corresponding to the channel pixel change curve, and select the source signal with the highest Pearson correlation coefficient in the green channel as the rPPG signal.

[0040] Noise removal was performed on the acquired single-channel facial pixels. After obtaining the single-channel pixel light variation curves of the face, the effective heartbeat information was easily obscured and interfered with by the large amount of noise in the signal. The FastICA algorithm was used to separate the source signals contained in the signal, and the source signal with the highest Pearson coefficient in the green channel was extracted as the rPPG signal related to heartbeat. To further remove high-frequency noise and baseline drift, wavelet transform was used to reconstruct the signal within the normal heartbeat frequency range. The effective pulse wave frequency in the signal was obtained through Fast Fourier Transform, and a narrowband pass filter was set to obtain a clean rPPG signal.

[0041] This step also includes steps S31 to S33.

[0042] Step S31: Construct an observation matrix in the time dimension using the channel pixel change curve, and perform whitening processing on the observation matrix to obtain a whitening matrix.

[0043] In this step, the pixel change curves of the R, G, and B channels are aligned with a uniform sampling rate and timestamp, and an analysis window of length L is extracted to construct the original observation matrix X∈R^{M×L}, where M is the number of channels. Each channel of X is detrended, mean-reduced, and its amplitude is normalized to form a zero-mean matrix X0. Based on X0, the observation covariance matrix C is estimated, and its eigenvalue decomposition or singular value decomposition is performed to obtain the whitening transformation matrix W. A linear transformation of X0 using W yields the whitening matrix Z=W·X0, and abnormal eigenvalues ​​are thresholded to ensure numerical stability.

[0044] Step S32: Perform blind source separation on the whitening matrix to obtain multiple source signal components. Based on physiological priors and quality indicators, select source signal components related to heartbeat from all source signal components, label them, and use them as candidate source signals. Label the remaining unlabeled source signal components as respiratory, light change, motion artifacts, or noise components according to their spectrum and correlation.

[0045] In this step, independent component analysis (ICA) is applied to the whitening matrix Z to obtain the source signal matrix S and the corresponding unmixing matrix A. Power spectral density, peak frequency, in-band energy, periodicity, and autocorrelation characteristics are calculated for each source signal component. Based on the feasible heart rate band, physiological harmonic structure, and signal quality indicators, heartbeat-related components are selected and labeled as candidate source signals. Based on spectral position, bandwidth, low-frequency trend, and correlation with external / common-mode references, the remaining components are labeled as respiration, illumination changes, motion artifacts, or noise. A confidence score is generated for each label. When multiple heartbeat candidates exist, the first K are retained for subsequent fusion.

[0046] Furthermore, blind source separation includes any one of independent component analysis, nonnegative matrix factorization, principal component analysis combined with independent rotation, and sparse component analysis.

[0047] Step S33: Reconstruct the candidate source signals in time series based on the separation matrix to obtain multiple source signals corresponding to the channel pixel change curves.

[0048] In this step, based on the whitening matrix, separation matrix, and mixing matrix obtained in steps S31 and S32, the scaling and sign are unified to eliminate the inherent scaling and arrangement uncertainty of ICA; labeled cardiac candidate components are selected from the source signal matrix, sorted by confidence level, and one or more component sets to be reconstructed are determined; the mixing matrix is ​​used to map the candidate source signals back to the observation channel domain to obtain the reconstructed time series corresponding to the original R / G / B channels; when there are multiple candidate components, they are weighted and fused according to confidence level or SNR, and frame / window level quality indicators are output.

[0049] Step S4: After spline interpolation of the rPPG signal, peak detection is performed to obtain the RR interval sequence, and the HRV feature set is calculated based on the RR interval sequence. The HRV feature set includes time domain features, frequency domain features, and nonlinear feature parameters.

[0050] HRV feature extraction was performed on the clean rPPG signal. Cubic spline interpolation was applied to the rPPG signal, upsampling to 240Hz, followed by peak point extraction. The RR interval sequence was obtained by calculating the time intervals between peak points. Outliers in the RR interval sequence were removed, and the HRV time-domain curve was plotted. Statistical methods were used to obtain the time-domain features, and frequency domain analysis was performed using Welch power spectrum plots to extract frequency domain feature parameters. Nonlinear feature parameters were extracted based on Poincaré scatter plot analysis.

[0051] This step also includes steps S41 to S43.

[0052] Step S41: Resample the rPPG signal along the time axis at a preset sampling rate to obtain a uniform sampling sequence.

[0053] In this step, a non-uniform time axis is constructed based on the timestamp sequence of the original rPPG signal, and a target sampling rate and a uniform time grid are set; the original rPPG signal is detrended and band-limited pre-filtered to limit the bandwidth to below the Nyquist frequency of the target sampling rate; the rPPG signal is resampled on the uniform time grid using band-limited interpolation or spline interpolation to obtain a uniform sampling sequence; missing frames and edge samples are robustly processed and resampling quality indicators are output.

[0054] Step S42: Perform local maximum detection on the filtered signal obtained after filtering the uniform sampling sequence to obtain a set of candidate peaks.

[0055] In this step, the uniformly sampled sequence is subjected to cardiac frequency bandpass filtering and amplitude standardization to obtain a filtered signal; local maxima are detected on the filtered signal and amplitude thresholds and minimum peak spacing constraints are applied; the peak set is screened for minimum / maximum peak spacing, rhythm consistency and noise suppression by combining cardiac a priori or upper window estimation; double peaks that are too close and shoulder peaks are merged, abnormal peaks are removed, and candidate peak set and quality indicators are output.

[0056] Step S43: Perform parabolic fitting in the neighborhood of each candidate peak in the candidate peak set to obtain the candidate peak position, and calculate the RR interval sequence based on the time difference between any two adjacent selected peak positions.

[0057] In this step, a time neighborhood with a fixed or adaptive width is constructed around the candidate peak index, and the neighborhood signal is smoothed with zero phase. A quadratic function is fitted in the neighborhood using the least squares method, and its extreme points are used as the precise peak positions and peak values. The fitting curvature, peak-to-valley difference, and consistency with the expected heart rate cycle are verified. If adjacent peaks overlap or their order is abnormal, they are corrected according to the rules. The RR interval sequence is obtained based on the time difference between adjacent peaks, and robust statistics and quality indicators are output.

[0058] Step S5: Based on the HRV feature set and TCM exterior-interior syndrome labels, construct a syndrome identification model. Input the HRV feature set of the test subject into the syndrome identification model for discrimination to obtain the TCM exterior-interior syndrome identification results of the test subject.

[0059] This step also includes steps S51 to S52.

[0060] Step S51: Obtain the HRV feature set of the object to be tested, and align the HRV feature set with the same order and names of the fields in the same training stage to obtain the preprocessed features.

[0061] In this step, outlier identification and repair are performed based on the RR interval sequence from step S43 to obtain a cleaned RR sequence; HRV feature calculation: HRV features defined by the preset feature dictionary are calculated in the time domain, frequency domain, and nonlinear domain; the current feature set is aligned to a unified vector space according to the feature field names and order in the training phase, and missing terms are completed using regularization; the aligned features are scaled and encoded according to the statistics in the training phase, and preprocessed features for model inference are output.

[0062] Step S52: Input the preprocessed features into the syndrome identification model to obtain the probability distribution of each syndrome category. Based on the preset discrimination threshold, map the probability distribution of each syndrome category to the final syndrome category. In the case of multi-classification, select the category corresponding to the largest posterior probability to output the TCM exterior-interior syndrome identification result.

[0063] In one example, based on the heart rate variability indicators and parameters corresponding to different TCM exterior-interior syndromes, potential TCM exterior-interior syndrome names and knowledge are determined, and a TCM exterior-interior syndrome knowledge base is constructed.

[0064] For example, exterior cold syndrome reflects an early pathological state in which cold pathogens invade the exterior and suppress the defensive yang. It is closely related to autonomic nervous system activity, especially under the influence of cold pathogens, the body initiates a stress response through the autonomic nervous system. In exterior cold syndrome, sympathetic nerve excitation is enhanced, manifesting as peripheral vasoconstriction, decreased skin temperature, increased heart rate, and muscle tension, causing symptoms such as chills, headache, and neck and back pain. Simultaneously, due to the cold pathogens binding the exterior and closing the pores, sweat gland secretion is suppressed, resulting in anhidrosis. This state of sympathetic hyperactivity and vagal nerve suppression also conforms to the physiological mechanism of "cold stimulation triggering a sympathetic response" in modern medicine. Clinically, if exterior cold syndrome is not relieved and the cold is dispelled in time, it may lead to autonomic nervous system dysfunction, further affecting body temperature regulation, immune response, and endocrine status.

[0065] Specifically, in the time domain analysis of heart rate variability (HRV) for cold syndrome: RMSSD (15-25ms) decreased, SDSD (40-55ms) increased, and SDNN (90-120ms) increased; in the frequency domain analysis: LF (ms²) (800-1500ms²) increased, HF (ms²) (200-400ms²) decreased, and LF / HF (3.0-4.5) increased; in the nonlinear analysis: SD1 (18-25ms) decreased, SD2 (100-130ms) increased, and SD1 / SD2 (0.18-0.25) decreased.

[0066] Establish an identification model to determine the relationship between patients' heart rate variability indicators and parameters and the corresponding names and knowledge of different TCM syndromes. Since different factors such as gender and age directly affect the results of heart rate variability measurement indicators and parameters, when establishing the identification model, it is also necessary to combine various indicators and parameters with gender, age, and other intervals to establish a joint identification model based on gender, age, and data intervals.

[0067] The KNN algorithm was used to determine the heart rate variability indicators and parameters of the patients under test, and to obtain the corresponding TCM syndrome names and knowledge.

[0068] Here, the specific method of using the KNN algorithm to judge and identify the heart rate variability indicators and parameters of the patient to be tested is as follows: For each heart rate variability indicator and parameter of the patient, select the K elements closest to the heart rate variability indicators and parameters of the patient to be tested from the corresponding sample vector, obtain the interval range to which the K elements belong in the identification model, and count the mode of the interval range to which the K elements belong. Use the interval range where the mode is located as the interval range of the heart rate variability indicators and parameters of the patient to be tested, where K=3; if the heart rate variability indicators and parameters of the patient to be tested meet the characteristic information of a certain syndrome, then it is identified as a certain syndrome of TCM exterior-interior, and the corresponding TCM exterior-interior syndrome name and knowledge are obtained; otherwise, it is not.

[0069] In this embodiment, the facial ROI is extracted and the mean pixel values ​​of the RGB three channels are calculated during the preprocessing stage to construct channel pixel change curves. This effectively suppresses random noise and local texture interference. Source signal separation separates physiological components from illumination changes, head movements, and camera noise, improving the signal-to-noise ratio of pulse-related components. Furthermore, the source with the highest Pearson correlation coefficient with the green channel is used as the rPPG signal. Leveraging the green channel's greater sensitivity to blood volume changes, the accuracy and robustness of cardiac information extraction are further improved. Peak detection is performed after spline interpolation to smooth out the discrete errors caused by irregular sampling. This method improves the stability of heartbeat localization and RR interval estimation, reduces the false negative rate, and systematically extracts time-domain, frequency-domain, and nonlinear features from the RR interval sequence reconstructed by rPPG to form a feature set covering the multidimensional mechanisms of autonomic nervous system regulation. This can more sensitively characterize changes in sympathetic / parasympathetic balance, stress load, rhythmicity, and complexity, and improve the ability of the syndrome identification model to perceive subtle differences in physiological states. By supervising the learning of HRV features and TCM exterior-interior syndrome labels, a mapping relationship from physiological signals to syndrome representation is established, reducing the subjectivity and consistency problems caused by relying on pure experience judgment and improving the stability of diagnosis.

[0070] Based on the above method, this application discloses a non-contact TCM syndrome differentiation system based on HRV data, referencing... Figure 2 The non-contact TCM syndrome differentiation system 1 includes a data acquisition module 11, a data processing module 12, and a syndrome differentiation module 13, wherein... Data acquisition module 11 is used to perform non-contact facial video acquisition of the test object through remote optical volumetric tracing, and to acquire raw facial video. The data processing module 12 is used to preprocess each frame of facial image in the original facial video, obtain the average ROI pixel value corresponding to the RGB three channels in the facial image, and form a channel pixel change curve based on the average ROI pixel value. The source signal of the channel pixel change curve is separated to obtain multiple source signals corresponding to the channel pixel change curve. The source signal with the highest Pearson correlation coefficient in the green channel is selected as the rPPG signal. After spline interpolation of the rPPG signal, peak detection is performed to obtain the RR interval sequence. The HRV feature set is calculated based on the RR interval sequence. The HRV feature set includes time domain features, frequency domain features and nonlinear feature parameters. The syndrome identification module 13 is used to construct a syndrome identification model based on the HRV feature set and TCM exterior-interior syndrome labels. The HRV feature set of the test subject is input into the syndrome identification model for discrimination to obtain the TCM exterior-interior syndrome identification results of the test subject.

[0071] In one example, the data processing module 12 performs face detection and facial keypoint localization on each frame of the original facial video to obtain face bounding boxes and facial keypoints. The face bounding boxes include multiple facial anatomical regions. Based on the facial keypoints in the multiple facial anatomical regions within the face bounding boxes, the region of interest (ROI) corresponding to the facial anatomical regions is determined. Background removal of non-skin regions in the ROI is performed through foreground segmentation to obtain a standard ROI. Affine registration is performed on the standard ROIs in adjacent frames of facial images, and the standard ROI in the current frame of facial images is aligned to the reference frame. Within the standard ROI in each frame of facial images, the pixel intensity statistics for the RGB three channels are calculated to obtain the average pixel value of the RGB three channels in the facial image.

[0072] In one example, the data processing module 12 is used to construct an observation matrix in the time dimension based on the channel pixel change curve, and to whiten the observation matrix to obtain a whitening matrix; to perform blind source separation on the whitening matrix to obtain multiple source signal components, to select source signal components related to heartbeat from all source signal components based on physiological priors and quality indicators, to label them as candidate source signals, and to label the remaining unlabeled source signal components as respiratory, illumination change, motion artifacts or noise components according to their spectrum and correlation; and to reconstruct the candidate source signals in time series based on the separation matrix to obtain multiple source signals corresponding to the channel pixel change curve.

[0073] In one example, blind source separation includes any one of independent component analysis, nonnegative matrix factorization, principal component analysis combined with independent rotation, and sparse component analysis.

[0074] In one example, the data processing module 12 is used to resample the rPPG signal on the time axis at a preset sampling rate to obtain a uniform sampling sequence; to perform local maximum detection on the filtered signal obtained after filtering the uniform sampling sequence to obtain a set of candidate peaks; to perform parabolic fitting in the neighborhood of each candidate peak in the set of candidate peaks to obtain the position of the candidate peak, and to calculate the RR interval sequence based on the time difference between any two adjacent selected peak positions.

[0075] In one example, the syndrome identification module 13 is used to obtain the HRV feature set of the test object, align the HRV feature set with the same order and same name fields in the training stage to obtain preprocessed features; input the preprocessed features into the syndrome identification model to obtain the probability distribution of each syndrome category, map the probability distribution of each syndrome category to the final syndrome category according to the preset discrimination threshold, and select the category corresponding to the largest posterior probability in the multi-class case to output the TCM exterior-interior syndrome identification result.

[0076] In one example, the facial anatomical region is a skin region including at least one of the forehead region and the cheekbone region, the skin region being defined within the face bounding box based on facial key points; or the facial anatomical region is the forehead region located between the brow ridge and the hairline and / or the cheekbone region located between the lower edge of the eye socket, the outer side of the nasal ala and the cheekbone.

[0077] Please see Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device 2 may include: at least one processor 21, at least one network interface 24, user interface 23, memory 25, and at least one communication bus 22.

[0078] The communication bus 22 is used to enable communication between these components.

[0079] The user interface 23 may include a display screen and a camera. Optionally, the user interface 23 may also include a standard wired interface and a wireless interface.

[0080] The network interface 24 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0081] The processor 21 may include one or more processing cores. The processor 21 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 25, and by calling data stored in the memory 25. Optionally, the processor 21 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 21 and may be implemented as a separate chip.

[0082] The memory 25 may include random access memory (RAM) or read-only memory. Optionally, the memory 25 may include non-transitory computer-readable storage medium. The memory 25 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 25 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 25 may also be at least one storage device located remotely from the aforementioned processor 21. Figure 3 As shown, the memory 25, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a non-contact TCM syndrome differentiation method based on HRV data.

[0083] exist Figure 3In the electronic device 2 shown, the user interface 23 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 21 can be used to call the application program stored in the memory 25, which is a non-contact TCM syndrome differentiation method based on HRV data. When executed by one or more processors, the electronic device performs one or more methods as described in the above embodiments.

[0084] A computer-readable storage medium storing instructions that, when executed by one or more processors, cause a computer to perform one or more methods as described in the embodiments above.

[0085] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0086] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0090] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0091] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A non-contact TCM syndrome differentiation method based on HRV data, characterized in that, The method includes: Remote optical volumetric imaging is used to perform non-contact facial video acquisition of the test object, capturing the original facial video. Each frame of the facial image in the original facial video is preprocessed to obtain the average ROI pixel value corresponding to the RGB three channels in the facial image, and a channel pixel change curve is formed based on the average ROI pixel value. Source signal separation is performed on the channel pixel change curve to obtain multiple source signals corresponding to the channel pixel change curve, and the source signal with the highest Pearson correlation coefficient in the green channel is selected as the rPPG signal; After spline interpolation of the rPPG signal, peak detection is performed to obtain the RR interval sequence, and the HRV feature set is calculated based on the RR interval sequence, wherein the HRV feature set includes time domain features, frequency domain features and nonlinear feature parameters; Based on the HRV feature set and the TCM exterior-interior syndrome labels, a syndrome identification model is constructed. The HRV feature set of the test subject is input into the syndrome identification model for discrimination to obtain the TCM exterior-interior syndrome identification results of the test subject.

2. The non-contact TCM syndrome differentiation method based on HRV data as described in claim 1, characterized in that, The preprocessing of each frame of facial image in the original facial video specifically includes: Face detection and facial key point localization are performed on each frame of the original facial video to obtain face bounding boxes and facial key points, wherein the face bounding boxes include multiple facial anatomical regions; Based on the facial key points, the region of interest corresponding to the facial anatomical region in the face bounding box is determined, and the non-skin region in the region of interest is removed by foreground segmentation to obtain the standard region of interest. Affine registration is performed on the standard regions of interest in adjacent frame facial images, and the standard regions of interest in the current frame facial image are aligned to the reference frame. In each frame of the facial image, the pixel intensity statistics of the RGB three channels are calculated separately within the standard region of interest to obtain the average pixel value of the RGB three channels in the facial image.

3. The non-contact TCM syndrome differentiation method based on HRV data as described in claim 1, characterized in that, The step of separating the source signals of the channel pixel change curve to obtain multiple source signals corresponding to the channel pixel change curve specifically includes: An observation matrix is ​​constructed in the time dimension based on the channel pixel change curve, and the observation matrix is ​​whitened to obtain a whitening matrix; Blind source separation is performed on the whitening matrix to obtain multiple source signal components. Based on physiological priors and quality indicators, source signal components related to heartbeat are selected from all source signal components, labeled, and used as candidate source signals. The remaining unlabeled source signal components are labeled as respiratory, light change, motion artifacts, or noise components according to their spectrum and correlation. The candidate source signals are reconstructed in time series based on the separation matrix to obtain multiple source signals corresponding to the channel pixel change curve.

4. The non-contact TCM syndrome differentiation method based on HRV data as described in claim 3, characterized in that, The blind source separation includes any one of independent component analysis, nonnegative matrix factorization, principal component analysis combined with independent rotation, and sparse component analysis.

5. The non-contact TCM syndrome differentiation method based on HRV data as described in claim 1, characterized in that, The step of performing peak detection after spline interpolation of the rPPG signal to obtain the RR interval sequence specifically includes: The rPPG signal is resampled along the time axis at a preset sampling rate to obtain a uniform sampling sequence. Local maxima detection is performed on the filtered signal obtained after filtering the uniform sampling sequence to obtain a set of candidate peaks; Parabolic fitting is performed in the neighborhood of each candidate peak in the candidate peak set to obtain the candidate peak position, and the RR interval sequence is calculated based on the time difference between any two adjacent selected peak positions.

6. The non-contact TCM syndrome differentiation method based on HRV data as described in claim 1, characterized in that, The step of inputting the HRV feature set of the subject to be tested into the syndrome identification model for discrimination specifically includes: Obtain the HRV feature set of the object to be tested, and align the HRV feature set with the same order and the same name fields according to the training stage to obtain preprocessed features; The preprocessed features are input into the syndrome identification model to obtain the probability distribution of each syndrome category. Based on the preset discrimination threshold, the probability distribution of each syndrome category is mapped to the final syndrome category. In the case of multi-classification, the category corresponding to the largest posterior probability is selected to output the TCM exterior-interior syndrome identification result.

7. The non-contact TCM syndrome differentiation method based on HRV data as described in claim 2, characterized in that, The facial anatomical region is a skin region including at least one of the forehead region and the cheekbone region, and the skin region is determined within the face bounding box based on facial key points. Alternatively, the facial anatomical region may be the forehead region located between the brow bone and the hairline, and / or the cheekbone region located between the lower edge of the eye socket, the outer side of the nasal wing, and the cheekbone arch.

8. A non-contact TCM syndrome differentiation system based on HRV data, characterized in that, The non-contact TCM syndrome differentiation system (1) includes a data acquisition module (11), a data processing module (12), and a syndrome differentiation module (13), wherein, The data acquisition module (11) is used to perform non-contact facial video acquisition of the test object through remote optical volume tracing, and to acquire the original facial video. The data processing module (12) is used to preprocess each frame of facial image in the original facial video, obtain the average value of ROI pixels corresponding to the RGB three channels in the facial image, form a channel pixel change curve according to the average value of ROI pixels, perform source signal separation on the channel pixel change curve to obtain multiple source signals corresponding to the channel pixel change curve, select the source signal with the highest Pearson correlation coefficient in the green channel as the rPPG signal, perform spline interpolation on the rPPG signal and perform peak detection to obtain the RR interval sequence, and calculate the HRV feature set according to the RR interval sequence, wherein the HRV feature set includes time domain features, frequency domain features and nonlinear feature parameters; The syndrome identification module (13) is used to construct a syndrome identification model based on the HRV feature set and the TCM exterior-interior syndrome labels, and input the HRV feature set of the test subject into the syndrome identification model for discrimination, so as to obtain the TCM exterior-interior syndrome identification result of the test subject.

9. An electronic device, characterized in that, The device includes a processor (21), a memory (25), a user interface (23), and a network interface (24). The memory (25) is used to store instructions. The user interface (23) and the network interface (24) are used to communicate with other devices. The processor (21) is used to execute the instructions stored in the memory (25) to cause the electronic device (2) to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-7.

Images