Coronary heart disease data processing system based on traditional Chinese medicine inspection diagnosis model

Abstract

The invention discloses a coronary heart disease data processing system based on a traditional Chinese medicine inspection diagnosis model, and relates to the technical field of coronary heart disease data processing, and the system comprises a data collection unit which is used for collecting traditional Chinese medicine face diagnosis video data and western medicine dynamic electrocardiogram data; the feature extraction unit is used for extracting traditional Chinese medicine inspection physical signs from the video data and extracting electrocardio features from the electrocardio data, the traditional Chinese medicine inspection physical signs comprise sublingual collateral curvature dynamic change and tongue vibration frequency, and the electrocardio features comprise ST segment drift and R-R interval variation. According to the coronary heart disease data processing system based on the traditional Chinese medicine inspection diagnosis model, time-frequency synchronization is performed by combining the traditional Chinese medicine inspection diagnosis model and western medicine electrocardiogram monitoring data and utilizing a non-uniform sampling space-time diagram convolutional network, so that the problem of time-fundamental frequency mismatch between heterogenous data is solved, accurate alignment of the two types of data in a time-space domain is ensured, and the accuracy of coronary heart disease data processing is improved. And accurate support is provided for early diagnosis and risk assessment of the coronary heart disease.

Description

Technical Field

[0001] The present invention relates to the technical field of coronary heart disease data processing, and in particular to a coronary heart disease data processing system based on a traditional Chinese medicine visual diagnosis model. Background Art

[0002] The current system has some key bottleneck problems when integrating TCM visual signs (such as dynamic changes in sublingual collateral varicose veins) with Western medicine monitoring data (dynamic ECG ST segment drift). Specifically: Western medicine monitoring equipment (such as Holter ECG) samples at millisecond level, while TCM facial diagnosis video acquisition is usually 25fps, resulting in the inability to couple the oscillation characteristics of physical signs (such as tongue tremor frequency of 0.5-2Hz) and ECG RR interval variation in a unified phase space for analysis; existing time series alignment methods (such as DTW dynamic time warping) will destroy the phase synchronization relationship in the TCM "pulse-symptom correspondence" theory, thereby causing the problem of temporal fundamental frequency mismatch of heterogeneous data. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] In view of the deficiencies of the prior art, the present invention provides a coronary heart disease data processing system based on the TCM visual diagnosis model, which solves the problem of time fundamental frequency mismatch between heterogeneous data.

[0005] (II) Technical solution

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a coronary heart disease data processing system based on a traditional Chinese medicine visual diagnosis model, the system comprising:

[0007] A data acquisition unit, used to respectively acquire TCM facial diagnosis video data and Western medicine dynamic electrocardiogram data;

[0008] A feature extraction unit is used to extract TCM visual signs from the video data and extract ECG features from the ECG data. The TCM visual signs include dynamic changes in the varicose veins of the sublingual veins and the frequency of tongue tremor. The ECG features include ST segment drift and RR interval variation.

[0009] The non-uniform sampling spatiotemporal graph convolutional network unit includes a non-uniform time sampling module and a spatiotemporal graph construction module. The non-uniform time sampling module uses Legendre polynomial interpolation to perform multi-scale time domain reconstruction on ECG data to form a time-frequency representation that matches the time nodes of the video frame; the spatiotemporal graph construction module constructs a graph structure with the spatial nodes of TCM signs and the corresponding time information, and performs joint mapping with the phase information of the ECG data;

[0010] The data fusion and risk assessment unit performs fusion analysis based on aligned TCM physical signs and ECG characteristics, and outputs coronary heart disease risk classification and early warning results;

[0011] A storage unit, used to store original collected data, synchronously processed data and fusion analysis results;

[0012] Among them, the data fusion and risk assessment unit adopts a hybrid data fusion algorithm to dynamically weighted fuse the TCM physical signs and ECG characteristics synchronized by the non-uniform sampling spatiotemporal graph convolutional network unit, and generates coronary heart disease risk grading and early warning indicators through an adaptive correction mechanism.

[0013] Preferably, the non-uniform time sampling module uses Legendre polynomial interpolation to perform multi-scale time domain reconstruction on the ECG data to form a time-frequency representation that matches the time node of the video frame: the collected ECG data is filtered and denoised to ensure that the signal is stable so that the subsequent interpolation processing can accurately reflect the original signal characteristics; the target time node is determined according to the frame rate of the TCM facial diagnosis video data acquisition, and the target time node corresponds to the acquisition moment of the video frame to form a unified time base; the ECG data is segmented according to a predetermined time window, and a multi-scale analysis method is used in each segment to extract different frequency components of the signal, thereby providing a basis for subsequent multi-scale reconstruction; the ECG data in each segment is fitted using the Legendre polynomial interpolation method to calculate the corresponding reconstructed signal value, and the signal values ​​reconstructed at different scales are integrated to form a time-frequency representation that reflects both time information (aligned with the video frame) and frequency domain characteristics, which can reveal the dynamic changes of the ECG signal in each frequency component.

[0014] The space-time graph construction module forms a graph structure with the spatial nodes of TCM signs and the corresponding time information, and performs joint mapping with the phase information of the electrocardiogram data, including: extracting key signs from TCM face-to-face diagnosis video data, the key signs include the degree of varicose veins under the tongue and the frequency of tongue body tremor, and determining the spatial coordinates of the key signs in the image; associating each sign with the timestamp of the video frame in which it is located, and forming an initial node with time stamp information;

[0015] Based on the node information extracted above, a preliminary graph structure is constructed, in which each node represents the collection result of a TCM sign at a specific time point. According to the spatial proximity and temporal continuity between the nodes, the edges or connection relationships between the nodes are defined to form a space-time graph;

[0016] The phase information of ECG data is extracted from the time-frequency representation to form a set of phase values ​​corresponding to the timestamp. For each node in the preliminary space-time graph, the closest phase value is found in the ECG data phase information according to its timestamp. The found phase value is used as an additional attribute and combined with the feature vector of the node to complete the fusion mapping of TCM vital sign data and ECG phase information.

[0017] Preferably, the non-uniformly sampled spatiotemporal graph convolutional network (NU-STGCN) unit further includes a multi-scale phase correlation correction submodule, which performs local alignment of the TCM video data and the ECG data at different time scales to ensure that the inherent phase synchronization of the two data is maintained during the interpolation process, thereby overcoming the defect of the traditional DTW method that destroys the phase relationship in the "pulse-symptom correspondence" theory;

[0018] The multi-scale phase correlation correction submodule adopts a two-stage Legendre polynomial interpolation strategy: a primary coarse alignment stage, using low-order Legendre polynomials to obtain preliminary time correspondence;

[0019] In the fine correction stage, high-order Legendre polynomials are used to perform local fine-tuning based on the preliminary alignment to ensure high-precision matching of the two data sources in the time and frequency domains.

[0020] Preferably, the process of locally aligning the TCM video data and the ECG data at different time scales by the multi-scale phase correlation correction submodule includes: segmenting the ECG data and the TCM video data after non-uniform time sampling processing according to preset overlapping windows; performing noise suppression and normalization processing on each data segment to ensure that each data segment has a good signal-to-noise ratio in the frequency domain;

[0021] For each data segment, multi-scale decomposition technology is used to extract frequency domain information and phase information from ECG data and video data, respectively, and phase spectra at different scales are obtained, providing a basis for subsequent phase correlation calculations.

[0022] For each data segment at each scale, the frequency spectrum of the two sets of data is calculated using Fourier transform, and then the phase correlation function is obtained by calculating the normalized cross spectrum to determine the phase difference between the two data segments; the local time offset, that is, the optimal alignment position of the two data at this scale, is preliminarily determined using the peak position of the phase correlation function;

[0023] At low scale, low-order Legendre polynomial interpolation is used to fit the initially determined time offset to obtain a set of coarse alignment parameters, thereby achieving a preliminary time correspondence between the video data and the ECG data;

[0024] At high scales, high-order Legendre polynomial interpolation is used to locally fine-tune the primary coarse alignment results to further accurately determine the optimal alignment time point for each data segment; this stage ensures that the inherent phase information of the two data is retained during the fine alignment process to avoid losing key signal features due to over-smoothing;

[0025] The local alignment parameters obtained at different scales are integrated to form an overall local correction scheme.

[0026] Preferably, the non-uniform sampling spatiotemporal graph convolutional network unit is further provided with an adaptive sampling parameter optimization module, which dynamically adjusts the Legendre interpolation parameters through a machine learning method based on historical synchronous sample data to optimize the non-uniform sampling strategy;

[0027] The process of optimizing the non-uniform sampling strategy is as follows:

[0028] Collect historical sample data from the previously completed synchronization process. The historical sample data includes the reconstruction results of the ECG signal after preliminary Legendre interpolation, the time nodes corresponding to the video data, and the synchronization error between the two. Extract key features, including phase matching error, local signal fluctuation amplitude, and signal noise level. These features can reflect the effect of the current sampling and interpolation strategy.

[0029] Construct a training data set based on key features and associate historical sample data with the Legendre interpolation parameters used at the time; select a machine learning algorithm to train a prediction model that outputs the optimal Legendre interpolation parameters based on the features of the current data;

[0030] In the new round of data collection and synchronization, the synchronization error and key features of the current data segment are calculated in real time, and these real-time features are input into the trained prediction model to obtain the dynamically adjusted Legendre interpolation parameters;

[0031] The new parameters output by the model are used to update the non-uniform sampling strategy, that is, the updated Legendre polynomial interpolation parameters are used in the current data segment to reconstruct the ECG data in the multi-scale time domain.

[0032] Preferably, the TCM physical sign extraction part in the feature extraction unit adopts a preprocessing enhancement algorithm to amplify the low-frequency oscillation information in the video signal, and extracts the sublingual collaterals and tongue micro-motion features in combination with a convolutional neural network (CNN). The process includes: preliminary preprocessing of the collected TCM facial diagnosis video data, including image denoising, normalization and color correction, to eliminate illumination and background noise, laying the foundation for subsequent feature enhancement; for low-frequency oscillations that may exist in the video sequence, using time domain or frequency domain filtering methods to extract the corresponding low-frequency components in the video; amplifying the extracted low-frequency signal, commonly used methods include amplitude normalization, contrast enhancement and local dynamic range expansion, to ensure that this part of the information is more prominent in subsequent processing; fusing the filtered and amplified low-frequency oscillation information with the original video frame to generate a set of enhanced video sequences, which retain the original details while focusing on the sublingual area and tongue micro-motion information;

[0033] The enhanced video sequence is input into a pre-designed and trained CNN network; the CNN network automatically extracts local features through multiple layers of convolution, pooling and activation layers, focusing on capturing the vascular distribution, texture changes and subtle movement information of the tongue in the sublingual collateral area; the subsequent fully connected layer or feature fusion layer in the network further integrates the local features output by the convolution layer to form a high-dimensional feature vector that can characterize the TCM physical signs, and the high-dimensional feature vector extracted by CNN is output and passed to the subsequent data fusion and risk assessment module for alignment with ECG data and coronary heart disease risk assessment.

[0034] Preferably, the hybrid data fusion algorithm adopted by the data fusion and risk assessment unit combines Kalman filtering and statistical learning to dynamically weighted fuse the TCM physical signs and ECG characteristics synchronized by the non-uniformly sampled spatiotemporal graph convolutional network unit, and generate coronary heart disease risk grading and early warning indicators through an adaptive correction mechanism. During the fusion process, the hybrid data fusion algorithm sets a compensation factor for the time baseband mismatch problem;

[0035] The process is as follows: input the data synchronized by the non-uniform sampling spatiotemporal graph convolutional network (NU-STGCN) unit into the system, which includes the processed TCM physical signs and ECG features; normalize and suppress the noise of the input data to ensure that the data has stable statistical characteristics before fusion; build an initial state estimation model for TCM physical signs and ECG features based on historical synchronized sample data; construct a state vector containing two sets of features, and set the initial covariance matrix to provide a prediction basis for subsequent dynamic updates;

[0036] Use statistical learning methods (such as linear regression or support vector regression) to model historical data and learn the dynamic weight distribution of two types of features at different times; assign preliminary weights to TCM physical signs and ECG features based on the output of the statistical model to provide a reference for subsequent dynamic weighted fusion; calculate the compensation factor by comparing the error between the actual synchronization data and the theoretical model to address the time baseband mismatch problem caused by sampling frequency mismatch; use error statistics to determine the compensation factor and adjust the subsequent weight calculation so that the two sets of data can fully compensate for the time baseband deviation when fused;

[0037] In the Kalman filter prediction process, the initial weights obtained by statistical learning are used in combination with compensation factors to dynamically calculate the weighted values ​​of TCM signs and ECG characteristics; the predicted values ​​of the Kalman filter are weighted and fused with the current observed values, and the weights are jointly determined by the statistical model and the compensation factors, thereby generating a fused risk state estimate; according to the error between the fused output and the actual monitoring data, the update parameters of the Kalman filter and the weight distribution in the statistical model are adjusted in real time; that is, a closed-loop feedback mechanism is adopted, and if a large deviation is detected, the compensation factor and dynamic weight are automatically updated to ensure that the entire system always maintains high-precision synchronization during long-term monitoring; the fused risk state estimate is compared with a pre-set threshold to generate a coronary heart disease risk grading and warning indicators, among which the coronary heart disease risk grading includes low risk, medium risk, and high risk; the risk grading and warning indicators are transmitted to subsequent display or storage modules for clinical decision support.

[0038] Preferably, the data acquisition unit includes an asynchronous data buffer module, which uses a built-in time correction algorithm to perform preliminary rough alignment of the 25fps video and the millisecond-level ECG data, providing a preprocessing basis for the fine time-frequency synchronization of the subsequent non-uniformly sampled spatiotemporal graph convolutional network unit;

[0039] The asynchronous data buffer module uses the built-in time correction algorithm to perform preliminary rough alignment of video and ECG data as follows:

[0040] The video data of traditional Chinese medicine face-to-face consultation and the dynamic ECG data of western medicine are collected at the same time, each with a system timestamp; the video data and ECG data are stored in independent buffers respectively, while retaining the original timestamp information to solve the problem of different acquisition frequencies and delays of the two types of data; the initial time offset value of the video and ECG data is determined by using the system clock or preset synchronization signal; the timestamps in the collected data are normalized to ensure that the time format of different data sources is consistent; the ECG data are grouped according to the corresponding time period according to the fixed time interval of video acquisition; for each time node of the video frame, representative data within the time period is selected from the ECG data (for example, using the mean or nearest neighbor value) to achieve rough time alignment; according to the preset synchronization signal or historical calibration data, the time offset of the two types of data is adjusted to ensure that the video frame and the ECG data are basically corresponding in the rough alignment stage, and the ECG data and video frames that have undergone rough alignment are combined according to the unified time mark to form a preliminary preprocessing data set. The module has a built-in feedback mechanism to continuously monitor the rough alignment error and update the built-in time correction algorithm parameters when necessary.

[0041] Preferably, the non-uniformly sampled spatiotemporal graph convolutional network unit is provided with a feedback adjustment submodule, which monitors the phase matching error of the synchronous output in real time, and feeds back the error information to the adaptive sampling parameter optimization module and the multi-scale phase correlation correction submodule, so as to dynamically adjust the resampling window and interpolation parameters, thereby ensuring that the two types of data always maintain high-precision synchronization during long-term monitoring;

[0042] The process is as follows:

[0043] The feedback regulation submodule monitors the synchronization data output by the non-uniformly sampled spatiotemporal graph convolutional network (NU-STGCN) in real time, calculates the phase matching error between the TCM vital sign data and the ECG data, and uses the preset phase error index to quantify the current alignment effect;

[0044] Compare the phase error calculated in real time with the set error threshold to determine whether the current synchronization state is within the ideal range; identify short-term or long-term deviation characteristics based on the error change trend to provide a basis for subsequent parameter adjustments;

[0045] The calculated phase matching error and its statistical characteristics are organized into feedback information; the feedback information is transmitted to the adaptive sampling parameter optimization module and the multi-scale phase correlation correction submodule respectively;

[0046] After receiving the feedback, the adaptive sampling parameter optimization module uses machine learning or adaptive algorithms to adjust the interpolation parameters and resampling window size according to the current error level to optimize the time domain reconstruction of the ECG data; the multi-scale phase correlation correction submodule recalculates the phase correlation function in the local area based on the feedback information, and dynamically adjusts the interpolation strategy of high-order and low-order Legendre polynomials to achieve more precise local alignment; the adjusted parameters are immediately applied to subsequent data processing, and the feedback regulation submodule continues to monitor the new synchronization output to form a closed-loop control; through continuous error monitoring and parameter updating, it is ensured that during long-term monitoring, the two types of data always maintain a high-precision time-frequency synchronization state.

[0047] A computer-readable storage medium stores computer program codes, which, when executed by a processor, can implement all or part of the functions of a coronary heart disease data processing system based on a traditional Chinese medicine visual diagnosis model.

[0048] (III) Beneficial effects

[0049] The present invention provides a coronary heart disease data processing system based on the traditional Chinese medicine visual diagnosis model. It has the following beneficial effects:

[0050] This coronary heart disease data processing system based on the TCM visual diagnosis model combines the TCM visual diagnosis model with Western medicine electrocardiogram monitoring data, and uses the non-uniform sampling spatiotemporal graph convolutional network (NU-STGCN) for time-frequency synchronization, which solves the problem of temporal fundamental frequency mismatch between heterogeneous data and ensures the precise alignment of the two types of data in the time and space domains. Finally, the system comprehensively analyzes the characteristics of TCM and Western medicine through a data fusion algorithm, providing accurate support for the early diagnosis and risk assessment of coronary heart disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the overall framework of the present invention;

[0052] Figure 2 It is a schematic diagram of the process of the non-uniform sampling spatiotemporal graph convolutional network unit of the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] See also Figure 1 and Figure 2 The present invention provides a technical solution: a coronary heart disease data processing system based on a traditional Chinese medicine visual diagnosis model, the system comprising:

[0055] A data acquisition unit, used to respectively acquire TCM facial diagnosis video data and Western medicine dynamic electrocardiogram data;

[0056] A feature extraction unit is used to extract TCM physical signs from video data and extract ECG features from ECG data. TCM physical signs include dynamic changes in the varicose veins of the sublingual veins and the frequency of tongue tremor. ECG features include ST segment drift and RR interval variation.

[0057] The non-uniform sampling spatiotemporal graph convolutional network (NU-STGCN) unit includes a non-uniform time sampling module and a spatiotemporal graph construction module. The non-uniform time sampling module uses Legendre polynomial interpolation to perform multi-scale time domain reconstruction on ECG data to form a time-frequency representation that matches the time nodes of the video frame; the spatiotemporal graph construction module constructs a graph structure with the spatial nodes of TCM signs and the corresponding time information, and jointly maps it with the phase information of the ECG data;

[0058] The data fusion and risk assessment unit performs fusion analysis based on aligned TCM physical signs and ECG characteristics, and outputs coronary heart disease risk classification and early warning results;

[0059] A storage unit, used to store original collected data, synchronously processed data and fusion analysis results;

[0060] Among them, the data fusion and risk assessment unit adopts a hybrid data fusion algorithm to dynamically weighted fuse the TCM physical signs and ECG characteristics synchronized by the non-uniform sampling spatiotemporal graph convolutional network unit, and generates coronary heart disease risk grading and early warning indicators through an adaptive correction mechanism.

[0061] The non-uniform time sampling module uses Legendre polynomial interpolation to perform multi-scale time domain reconstruction on ECG data to form a time-frequency representation that matches the time nodes of the video frame: the collected ECG data is filtered and denoised to ensure that the signal is stable so that the subsequent interpolation processing can accurately reflect the characteristics of the original signal; according to the frame rate of the TCM facial diagnosis video data acquisition, about 25fps, a series of target time nodes are determined, and the target time nodes correspond to the acquisition moments of the video frames to form a unified time base; the ECG data is segmented according to the predetermined time window, and a multi-scale analysis method is used in each segment, such as wavelet or empirical mode decomposition, to extract different frequency components of the signal, thereby providing a basis for subsequent multi-scale reconstruction; The ECG data is fitted using the Legendre polynomial interpolation method to calculate the corresponding reconstructed signal values, and the signal values ​​reconstructed at different scales are integrated to form a time-frequency representation that reflects time information, aligns with video frames, and contains frequency domain features. This representation can reveal the dynamic changes of ECG signals in each frequency component. Finally, the formed time-frequency representation is used as output data for subsequent fusion analysis, coupling matching, and coronary heart disease risk assessment with TCM physical sign data. This process ensures that the millisecond ECG data is accurately aligned with the time nodes of the 25fps video frame after multi-scale reconstruction, while retaining the phase information of each frequency component, thus providing a solid foundation for the joint analysis of cross-modal data.

[0062] Among them, the process of fitting the ECG data in each segment using the Legendre polynomial interpolation method is as follows: selecting the appropriate polynomial order and determining the low-order and high-order interpolation strategies based on the data characteristics; using the least squares method to calculate the coefficients of the Legendre polynomials; at the time node corresponding to each target video frame, the corresponding reconstructed signal value is calculated based on the determined Legendre polynomials to achieve multi-scale time domain reconstruction of the ECG data.

[0063] The space-time graph construction module forms a graph structure by combining the spatial nodes of TCM signs with the corresponding time information, and performs joint mapping with the phase information of ECG data. The process includes: extracting key signs from TCM face-to-face diagnosis video data, including the degree of varicose veins under the tongue and the frequency of tongue tremor, and determining the spatial coordinates of the key signs in the image; associating each sign with the timestamp of the video frame in which it is located, and forming an initial node with time stamp information;

[0064] Based on the node information extracted above, a preliminary graph structure is constructed, in which each node represents the collection result of a TCM sign at a specific time point. According to the spatial proximity and temporal continuity between the nodes, the edge or connection relationship between the nodes is defined to form a space-time graph, in which the spatial proximity includes the adjacent relationship of anatomical regions;

[0065] The phase information of the ECG data is extracted from the time-frequency representation to form a set of phase values ​​corresponding to the timestamp; for each node in the preliminary space-time graph, the closest phase value is found in the ECG data phase information according to its timestamp; the found phase value is used as an additional attribute and combined with the feature vector of the node to complete the fusion mapping of the TCM physical sign data and the ECG phase information; through the above process, the space-time graph construction module can form a graph structure with the physical sign information extracted from the TCM facial consultation video and the corresponding time information, and jointly map the phase information of the ECG data to the nodes of the graph, thereby realizing the joint representation of cross-modal data in the time-frequency domain.

[0066] The non-uniformly sampled spatiotemporal graph convolutional network (NU-STGCN) unit further includes a multi-scale phase correlation correction submodule, which locally aligns the TCM video data and the ECG data at different time scales to ensure that the inherent phase synchronization of the two data is maintained during the interpolation process, thereby overcoming the defect of the traditional DTW method that destroys the phase relationship in the "pulse-symptom correspondence" theory;

[0067] The multi-scale phase correlation correction submodule adopts a two-stage Legendre polynomial interpolation strategy: in the primary coarse alignment stage, low-order Legendre polynomials are used to obtain preliminary time correspondences;

[0068] In the fine correction stage, high-order Legendre polynomials are used to perform local fine-tuning based on the preliminary alignment to ensure high-precision matching of the two data sources in the time and frequency domains.

[0069] The process of local alignment of TCM video data and ECG data at different time scales by the multi-scale phase correlation correction submodule includes: segmenting the ECG data and TCM video data after non-uniform time sampling according to the preset overlapping windows; performing noise suppression and normalization processing on each data segment to ensure that each data segment has a good signal-to-noise ratio in the frequency domain;

[0070] For each data segment, wavelet transform or multi-resolution Fourier transform is used to extract the frequency domain information and phase information in the ECG data and video data respectively, and the phase spectrum at different scales is obtained to provide a basis for subsequent phase correlation calculation;

[0071] For each data segment at each scale, the frequency spectrum of the two sets of data is calculated using Fourier transform, and then the phase correlation function is obtained by calculating the normalized cross spectrum to determine the phase difference between the two data segments; the local time offset, that is, the optimal alignment position of the two data at this scale, is preliminarily determined using the peak position of the phase correlation function;

[0072] Under the dominance of low-frequency components, low-order Legendre polynomial interpolation is used to fit the initially determined time offset to obtain a set of coarse alignment parameters, thereby achieving a preliminary time correspondence between video data and ECG data; under the details of high-frequency components, high-order Legendre polynomial interpolation is used to locally fine-tune the primary coarse alignment results to further accurately determine the optimal alignment time point for each data segment; this stage ensures that the inherent phase information of the two data is retained during the fine alignment process to avoid losing key signal features due to over-smoothing; the local alignment parameters obtained at different scales are integrated to form an overall local correction solution;

[0073] The feedback adjustment submodule monitors the phase matching error of each local area after alignment in real time, and dynamically adjusts the interpolation window and correction parameters to ensure high-precision phase synchronization during long-term monitoring.

[0074] Rejoin the data segments after local alignment correction to form the globally aligned ECG and video data time series; use statistical analysis and phase consistency verification methods to evaluate the global alignment effect to ensure that the final output data can accurately reflect the joint characteristics of the two data in the time and frequency domains;

[0075] Through the above steps, local alignment of TCM video data and ECG data at different time scales is achieved based on the multi-scale phase correlation correction submodule. The strategy of combining low-order and high-order Legendre polynomial interpolation is used to effectively ensure the phase synchronization of the data during the interpolation process, thereby overcoming the defect of the traditional DTW method that destroys the phase relationship in the "pulse-symptom correspondence" theory.

[0076] The non-uniform sampling spatiotemporal graph convolutional network unit is further provided with an adaptive sampling parameter optimization module, which dynamically adjusts the Legendre interpolation parameters based on historical synchronous sample data through machine learning methods to optimize the non-uniform sampling strategy;

[0077] The process of optimizing the non-uniform sampling strategy is as follows: collect historical sample data from the previously completed synchronization process, which includes the reconstruction results of the ECG signal after preliminary Legendre interpolation, the time nodes corresponding to the video data, and the synchronization errors between the two, such as phase deviation and interpolation error; extract key features, including phase matching error, local signal fluctuation amplitude, and signal noise level, which can reflect the effect of the current sampling and interpolation strategy;

[0078] Construct a training data set based on key features, and associate historical sample data with the Legendre interpolation parameters used at the time; select a regression model and train a prediction model that is used to output the optimal Legendre interpolation parameters based on the characteristics of the current data;

[0079] In the new round of data collection and synchronization, the synchronization error and key features of the current data segment are calculated in real time, and these real-time features are input into the trained prediction model to obtain the dynamically adjusted Legendre interpolation parameters;

[0080] The new parameters output by the model are used to update the non-uniform sampling strategy, that is, the updated Legendre polynomial interpolation parameters are used in the current data segment to reconstruct the ECG data in the multi-scale time domain; by updating the interpolation parameters, the alignment accuracy of the ECG data and the video data in the time and frequency domain is optimized, thereby achieving more accurate synchronization.

[0081] Compare the updated synchronization results with the expected alignment effect, and monitor the phase matching and interpolation errors in real time; send feedback information back to the adaptive sampling parameter optimization module to further adjust and fine-tune the model parameters to achieve closed-loop optimization and continuously improve the robustness and accuracy of the overall synchronization;

[0082] Through the above process, the adaptive sampling parameter optimization module can utilize the historical synchronization sample data and dynamically adjust the Legendre interpolation parameters through machine learning methods, thereby optimizing the non-uniform sampling strategy and achieving high-precision synchronization of ECG data and TCM video data in the time and frequency domain.

[0083] The TCM physical sign extraction part in the feature extraction unit adopts a preprocessing enhancement algorithm to amplify the low-frequency oscillation information in the video signal, and extracts the sublingual collaterals and tongue micro-motion features in combination with a convolutional neural network (CNN). The process includes: preliminary preprocessing of the collected TCM facial diagnosis video data, including image denoising, normalization and color correction, to eliminate illumination and background noise, laying the foundation for subsequent feature enhancement; for low-frequency oscillations that may exist in the video sequence, such as tongue vibration frequency in the range of 0.5 to 2 Hz, a bandpass filter is used to extract the corresponding low-frequency components in the video; the extracted low-frequency signal is amplified, and the commonly used methods include amplitude normalization, contrast enhancement and local dynamic range expansion to ensure that this part of the information is more prominent in subsequent processing; the low-frequency oscillation information after filtering and amplification is fused with the original video frame to generate a set of enhanced video sequences, which retain the original details while focusing on the sublingual area and tongue micro-motion information;

[0084] The enhanced video sequence is input into a pre-designed and trained CNN network; the CNN network automatically extracts local features through multiple layers of convolution, pooling and activation layers, focusing on capturing the vascular distribution, texture changes and subtle movement information of the sublingual collateral area, and the micro-motion of the tongue; the subsequent fully connected layer or feature fusion layer in the network further integrates the local features output by the convolution layer to form a high-dimensional feature vector that can characterize TCM physical signs, such as the degree of varicose veins of the sublingual collaterals and the oscillation micro-motion of the tongue; the high-dimensional feature vector extracted by CNN is output and passed to the subsequent data fusion and risk assessment module for alignment with ECG data and coronary heart disease risk assessment; through the above process, the preprocessing enhancement algorithm successfully amplifies the low-frequency oscillation information in the video signal, and combined with the feature extraction capability of CNN, it realizes the efficient capture of the sublingual collateral and tongue micro-motion features, providing accurate TCM physical sign data for the processing of coronary heart disease data based on the TCM visual diagnosis model.

[0085] The hybrid data fusion algorithm used by the data fusion and risk assessment unit combines Kalman filtering and statistical learning to dynamically weight the TCM signs and ECG features synchronized by the non-uniformly sampled spatiotemporal graph convolutional network unit, and generates coronary heart disease risk classification and early warning indicators through an adaptive correction mechanism. During the fusion process, the hybrid data fusion algorithm sets a compensation factor for the time baseband mismatch problem.

[0086] The process is as follows: input the data synchronized by the NU-STGCN unit into the system, which includes processed TCM physical signs (such as sublingual collateral varicose vein varicose veins, tongue micro-movement characteristics) and ECG characteristics (such as ST segment drift, RR interval variation); normalize and suppress noise on the input data to ensure that the data has stable statistical characteristics before fusion; build an initial state estimation model for TCM physical signs and ECG characteristics based on historical synchronized sample data; construct a state vector containing two sets of features, and set the initial covariance matrix to provide a prediction basis for subsequent dynamic updates;

[0087] Use linear regression or support vector regression to model historical data and learn the dynamic weight distribution of two types of features at different times; assign preliminary weights to TCM signs and ECG features based on the output of the statistical model to provide a reference for subsequent dynamic weighted fusion; calculate the compensation factor by comparing the error between the actual synchronization data and the theoretical model to address the time baseband mismatch problem caused by sampling frequency mismatch; determine the compensation factor using mean square error or variance change, and adjust subsequent weight calculations so that the two sets of data fully compensate for the time baseband deviation when fused;

[0088] In the Kalman filter prediction process, the initial weights obtained by statistical learning are used in combination with compensation factors to dynamically calculate the weighted values ​​of TCM physical signs and ECG characteristics; the predicted values ​​of the Kalman filter are weighted and fused with the current observed values, and the weights are jointly determined by the statistical model and the compensation factors, thereby generating a fused risk state estimate; according to the error between the fused output and the actual monitoring data, the update parameters of the Kalman filter and the weight distribution in the statistical model are adjusted in real time; that is, a closed-loop feedback mechanism is adopted, and if a large deviation is detected, the compensation factor and dynamic weight are automatically updated to ensure that the entire system always maintains high-precision synchronization during long-term monitoring; the fused risk state estimate is compared with a pre-set threshold to generate a coronary heart disease risk grading and warning indicators, where the coronary heart disease risk grading includes low risk, medium risk, and high risk; the risk grading and warning indicators are transmitted to subsequent display or storage modules for clinical decision support;

[0089] Through the above process, the hybrid data fusion algorithm fully combines the dynamic prediction advantages of Kalman filtering and the weight adjustment ability of statistical learning, and introduces compensation factors to effectively make up for the time base frequency mismatch caused by the mismatch of heterogeneous data sampling frequencies, thereby realizing the dynamic weighted fusion of TCM physical signs and electrocardiogram characteristics, and generating reliable coronary heart disease risk grading and early warning indicators.

[0090] The data acquisition unit includes an asynchronous data buffer module, which uses a built-in time correction algorithm to perform preliminary rough alignment of 25fps video and millisecond-level ECG data, providing a preprocessing basis for the fine time-frequency synchronization of the subsequent non-uniformly sampled spatiotemporal graph convolutional network unit;

[0091] The asynchronous data buffer module uses the built-in time correction algorithm to perform preliminary rough alignment of 25fps video and millisecond ECG data as follows:

[0092] Collect TCM face-to-face video data and Western medicine dynamic ECG data at the same time. The frame rate of TCM face-to-face video data is about 25fps, and the Western medicine dynamic ECG data is sampled at the millisecond level, each with a system timestamp. The video data and ECG data are stored in independent buffers, while retaining the original timestamp information to solve the problem of different acquisition frequencies and delays of the two types of data. Use the system clock or preset synchronization signal to determine the initial time offset value of the video and ECG data. Normalize the timestamps in the collected data to ensure that the time format of different data sources is consistent.

[0093] According to the 40ms / frame time interval of video acquisition, the ECG data are grouped according to the corresponding time periods; for each time node of the video frame, the representative data within the time period is selected from the ECG data, that is, the mean or nearest neighbor value is used to achieve rough time alignment; according to the preset synchronization signal or historical calibration data, the time offset of the two types of data is adjusted to ensure that the video frame and the ECG data are basically corresponding in the rough alignment stage; the ECG data and video frames that have undergone rough alignment are combined according to the unified time mark to form a preliminary preprocessing data set; this data set is used as the input basis for the subsequent non-uniform sampling spatiotemporal graph convolutional network (NU-STGCN) unit to perform fine time-frequency synchronization, ensuring that the subsequent processing stage can further extract accurate features on a unified time base;

[0094] The module has a built-in feedback mechanism to continuously monitor the coarse alignment error and update the built-in time correction algorithm parameters when necessary. Through statistical analysis of historical synchronization samples, the time correction strategy is further optimized to improve the coarse alignment accuracy and create good conditions for subsequent fine synchronization.

[0095] The asynchronous data buffer module uses the built-in time correction algorithm to perform preliminary rough alignment of the 25fps video and the millisecond-level ECG data, and finally generates a preprocessed data set under a unified time base, which provides a solid foundation for the subsequent fine time-frequency synchronization of the non-uniformly sampled spatiotemporal graph convolutional network units.

[0096] The non-uniform sampling spatiotemporal graph convolutional network unit is equipped with a feedback regulation submodule, which monitors the phase matching error of the synchronous output in real time and feeds back the error information to the adaptive sampling parameter optimization module and the multi-scale phase correlation correction submodule for dynamically adjusting the resampling window and interpolation parameters to ensure that the two data always maintain high-precision synchronization during long-term monitoring. The process is as follows: the feedback regulation submodule monitors the synchronous data output by the non-uniform sampling spatiotemporal graph convolutional network (NU-STGCN) in real time, calculates the phase matching error between the TCM physical sign data and the ECG data; the preset phase error indicators are used: the mean square error of the phase difference or the average phase offset, and the phase error is measured. The current alignment effect is optimized; the phase error calculated in real time is compared with the set error threshold to determine whether the current synchronization state is within the ideal range; according to the error change trend, the short-term or long-term deviation characteristics are identified to provide a basis for subsequent parameter adjustment, and the calculated phase matching error and its statistical characteristics are sorted into feedback information, which is transmitted to the adaptive sampling parameter optimization module and the multi-scale phase correlation correction submodule respectively. The adaptive sampling parameter optimization module is used to dynamically adjust the Legendre interpolation parameters and the non-uniform sampling strategy according to the historical and real-time errors; the multi-scale phase correlation correction submodule is used to further fine-tune the local alignment parameters at different scales.

[0097] After receiving the feedback, the adaptive sampling parameter optimization module uses machine learning or adaptive algorithms to adjust the interpolation parameters and resampling window size according to the current error level to optimize the time domain reconstruction of the ECG data; the multi-scale phase correlation correction submodule recalculates the phase correlation function in the local area based on the feedback information, and dynamically adjusts the interpolation strategy of high-order and low-order Legendre polynomials to achieve more precise local alignment; the adjusted parameters are immediately applied to subsequent data processing, and the feedback regulation submodule continues to monitor the new synchronization output to form a closed-loop control; through continuous error monitoring and parameter updating, it is ensured that the two types of data always maintain a high-precision time-frequency synchronization state during long-term monitoring;

[0098] Through the above-mentioned closed-loop feedback adjustment process, the feedback adjustment submodule effectively realizes the real-time monitoring of the synchronous output phase matching error, and feeds back the error information to the relevant optimization module, thereby dynamically adjusting the resampling window and interpolation parameters to ensure that the entire system always meets the requirements of precise synchronization during long-term monitoring.

[0099] A computer-readable storage medium stores computer program codes, which can realize all or part of the functions of a coronary heart disease data processing system based on a traditional Chinese medicine visual diagnosis model when executed by a processor.

[0100] This technical solution combines the TCM visual diagnosis model with Western medicine ECG monitoring data, and uses the non-uniform sampling spatiotemporal graph convolutional network (NU-STGCN) for time-frequency synchronization, which solves the problem of temporal fundamental frequency mismatch between heterogeneous data and ensures the precise alignment of the two types of data in the time and space domain. Finally, the system comprehensively analyzes the characteristics of TCM and Western medicine through a data fusion algorithm, providing accurate support for the early diagnosis and risk assessment of coronary heart disease.

[0101] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0102] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A coronary heart disease data processing system based on the traditional Chinese medicine visual diagnosis model, characterized in that: The system comprises: A data acquisition unit, used to respectively acquire TCM facial diagnosis video data and Western medicine dynamic electrocardiogram data; A feature extraction unit is used to extract TCM visual signs from the video data and extract ECG features from the ECG data. The TCM visual signs include dynamic changes in the varicose veins of the sublingual veins and the frequency of tongue tremor. The ECG features include ST segment drift and RR interval variation. The non-uniform sampling spatiotemporal graph convolutional network unit includes a non-uniform time sampling module and a spatiotemporal graph construction module. The non-uniform time sampling module uses Legendre polynomial interpolation to perform multi-scale time domain reconstruction on ECG data to form a time-frequency representation that matches the time nodes of the video frame; the spatiotemporal graph construction module constructs a graph structure with the spatial nodes of TCM signs and the corresponding time information, and performs joint mapping with the phase information of the ECG data; The data fusion and risk assessment unit performs fusion analysis based on aligned TCM physical signs and ECG characteristics, and outputs coronary heart disease risk classification and early warning results; A storage unit, used to store original collected data, synchronously processed data and fusion analysis results; Among them, the data fusion and risk assessment unit adopts a hybrid data fusion algorithm to dynamically weighted fuse the TCM physical signs and ECG characteristics synchronized by the non-uniform sampling spatiotemporal graph convolutional network unit, and generates coronary heart disease risk grading and early warning indicators through an adaptive correction mechanism.

2. A coronary heart disease data processing system based on the TCM visual diagnosis model according to claim 1, characterized in that: The non-uniform time sampling module uses Legendre polynomial interpolation to perform multi-scale time domain reconstruction on the ECG data to form a time-frequency representation that matches the video frame time node: filtering and denoising the collected ECG data, determining the target time node according to the frame rate of the TCM face diagnosis video data collection, segmenting the ECG data according to a predetermined time window, and using a multi-scale analysis method to extract different frequency components of the signal in each segment, fitting the ECG data in each segment using the Legendre polynomial interpolation method, calculating the corresponding reconstructed signal value, and integrating them to form a time-frequency representation; The space-time graph construction module forms a graph structure with the spatial nodes of TCM physical signs and the corresponding time information, and performs a joint mapping process with the phase information of ECG data, including: extracting key physical signs from TCM facial diagnosis video data, and determining the spatial coordinates of the key physical signs in the image; associating each physical sign with the timestamp of the video frame in which it is located to form an initial node; constructing a preliminary graph structure based on the node information, defining the edges or connection relationships between the nodes according to the spatial proximity and time continuity between the nodes, and forming a space-time graph; extracting the phase information of the ECG data from the time-frequency representation, for each node in the preliminary space-time graph, searching the closest phase value in the ECG data phase information according to its timestamp, and using the found phase value as an additional attribute and combining it with the feature vector of the node to complete the fusion mapping of TCM physical sign data and ECG phase information.

3. A coronary heart disease data processing system based on the TCM visual diagnosis model according to claim 2, characterized in that: The non-uniformly sampled spatiotemporal graph convolutional network unit further includes a multi-scale phase correlation correction submodule, which performs local alignment of the TCM video data and the ECG data at different time scales; The multi-scale phase correlation correction submodule adopts a two-stage Legendre polynomial interpolation strategy: a primary coarse alignment stage, using low-order Legendre polynomials to obtain preliminary time correspondence; In the fine correction stage, high-order Legendre polynomials are used to perform local fine adjustments based on the preliminary alignment.

4. A coronary heart disease data processing system based on the TCM visual diagnosis model according to claim 3, characterized in that: The process of the multi-scale phase correlation correction submodule locally aligning the TCM video data and the ECG data at different time scales includes: segmenting the ECG data and the TCM video data after non-uniform time sampling processing according to preset overlapping windows; for each data segment, using multi-scale decomposition technology to extract frequency domain information and phase information in the ECG data and the video data, respectively, to obtain phase spectra at different scales; for each data segment at each scale, using Fourier transform to calculate the frequency spectrum of the two sets of data, and obtaining the phase correlation function by calculating the normalized cross spectrum; using the peak position of the phase correlation function to preliminarily determine the local time offset; At low scales, low-order Legendre polynomial interpolation is used to fit the preliminarily determined time offset to obtain coarse alignment parameters. At high scales, high-order Legendre polynomial interpolation is used to locally fine-tune the primary coarse alignment results to determine the optimal alignment time point for each data segment. The local alignment parameters obtained at different scales are integrated to form a local correction scheme.

5. A coronary heart disease data processing system based on the TCM visual diagnosis model according to claim 4, characterized in that: The non-uniform sampling spatiotemporal graph convolutional network unit is further provided with an adaptive sampling parameter optimization module, which dynamically adjusts the Legendre interpolation parameters through a machine learning method based on historical synchronous sample data to optimize the non-uniform sampling strategy; The process of optimizing the non-uniform sampling strategy includes: collecting historical sample data from previously completed synchronization processes and extracting key features; A training data set is constructed based on key features, and the historical sample data is associated with the Legendre interpolation parameters used at the time. A machine learning algorithm is selected to train a prediction model, which is used to output the optimal Legendre interpolation parameters based on the characteristics of the current data. In a new round of data acquisition and synchronization, the synchronization error and key features of the current data segment are calculated in real time and input into the trained prediction model to obtain dynamically adjusted Legendre interpolation parameters. The updated Legendre polynomial interpolation parameters are used in the current data segment to reconstruct the ECG data in the multi-scale time domain.

6. A coronary heart disease data processing system based on the TCM visual diagnosis model according to claim 5, characterized in that: The TCM physical sign extraction part in the feature extraction unit adopts a preprocessing enhancement algorithm to amplify the low-frequency oscillation information in the video signal, and extracts the sublingual collaterals and tongue micro-movement features in combination with a convolutional neural network (CNN). The process includes: preliminary preprocessing of the collected TCM facial diagnosis video data, extracting the corresponding low-frequency components in the video using a time domain or frequency domain filtering method for the low-frequency oscillations that may exist in the video sequence, amplifying the extracted low-frequency signal, and fusing the filtered and amplified low-frequency oscillation information with the original video frame to generate a video sequence; inputting the video sequence into a pre-designed and trained CNN network, and the subsequent fully connected layer or feature fusion layer in the network further integrates the local features output by the convolution layer to form a high-dimensional feature vector, outputting the high-dimensional feature vector, and passing it to the data fusion and risk assessment module.

7. A coronary heart disease data processing system based on the TCM visual diagnosis model according to claim 6, characterized in that: The hybrid data fusion algorithm adopted by the data fusion and risk assessment unit combines Kalman filtering and statistical learning, dynamically weightedly fuses the TCM signs and ECG features synchronized by the non-uniform sampling spatiotemporal graph convolutional network unit, and generates coronary heart disease risk grading and early warning indicators through an adaptive correction mechanism. During the fusion process, the hybrid data fusion algorithm sets a compensation factor for the time baseband mismatch problem; the process is as follows: input the data synchronized by the non-uniform sampling spatiotemporal graph convolutional network unit into the system, construct an initial state estimation model for TCM signs and ECG features based on historical synchronized sample data, model the historical data using a statistical learning method, learn the dynamic weight distribution of the two types of features at different times, calculate the compensation factor by comparing the error between the actual synchronized data and the theoretical model, and in the Kalman filter prediction process, use the preliminary weights obtained by statistical learning and combine the compensation factor to calculate the weighted values ​​of TCM signs and ECG features, and adjust the update parameters of the Kalman filter and the weight distribution in the statistical model in real time according to the error between the fused output and the actual monitoring data; The fused risk status estimate is compared with the pre-set threshold to generate coronary heart disease risk grading and early warning indicators, where the coronary heart disease risk grading includes low risk, medium risk, and high risk.

8. The coronary heart disease data processing system based on the TCM visual diagnosis model according to claim 7 is characterized by: The data acquisition unit includes an asynchronous data buffer module, which uses a built-in time correction algorithm to perform preliminary rough alignment on the video and ECG data; the process is as follows: Chinese medicine facial diagnosis video data and Western medicine dynamic ECG data are collected simultaneously, each with a system timestamp; the video data and ECG data are stored in independent buffers respectively, while retaining the original timestamp information; the initial time offset value of the video and ECG data is determined using the system clock or a preset synchronization signal; the ECG data is grouped into corresponding time periods according to a fixed time interval for video acquisition; for each time node of a video frame, representative data within the time period is selected from the ECG data, and the time offset of the two types of data is adjusted according to a preset synchronization signal or historical calibration data, and the roughly aligned ECG data and video frames are combined according to a unified time mark to form a preliminary preprocessing data set.

9. A coronary heart disease data processing system based on the TCM visual diagnosis model according to claim 8, characterized in that: The non-uniformly sampled spatiotemporal graph convolutional network unit is provided with a feedback adjustment submodule, which monitors the phase matching error of the synchronous output in real time and feeds back the error information to the adaptive sampling parameter optimization module and the multi-scale phase correlation correction submodule for dynamically adjusting the resampling window and interpolation parameters.

10. A computer-readable storage medium, characterized in that: The storage medium stores computer program codes, which, when executed by a processor, can implement all or part of the functions of the coronary heart disease data processing system based on the TCM visual diagnosis model as described in any one of claims 1 to 9.

Images