Traditional Chinese medicine six-epimedium intelligent identification method and device fused with multi-modal data

Abstract

The invention provides a traditional Chinese medicine six-epimedium intelligent identification method and device fusing multi-modal data, and relates to the field of multi-modal intelligent medicine. The method comprises the following steps: acquiring multi-modal data of a patient suffering from six epidemic diseases of the traditional Chinese medicine; through feature extraction, facial optical physiological change features are obtained based on face video data, symptom features are extracted based on six-epidemic disease symptom data, and sympathetic nerve activity features are extracted based on the face video data and sympathetic nerve activity data; performing feature fusion on the preprocessed facial optical physiological change features, symptom features and sympathetic nerve activity features; inputting the fusion features into an interval judgment model established based on a KNN algorithm, and outputting a traditional Chinese medicine six-epimedium intelligent identification result according to the interval judgment model; and outputting a syndrome differentiation treatment scheme corresponding to the traditional Chinese medicine six-epimedium disease patient based on the traditional Chinese medicine six-epimedium intelligent identification result. The problem that a traditional Chinese medicine syndrome differentiation model is low in recognition precision in practical application is solved.

Description

Technical Field

[0001] This application relates to the field of multimodal intelligent medicine, and particularly to a method and device for intelligent identification of the six pathogenic factors in traditional Chinese medicine by integrating multimodal data. Background Art

[0002] The six pathogenic factors are a general term for six pathogenic evils, namely wind, cold, summer heat, dampness, dryness, and fire. The six pathogenic factors were originally six climatic factors in nature. When appropriate, they are normal "six qi", but when excessive, abnormal or the body's vital qi is insufficient, they become pathogenic factors, called "the six pathogenic factors". The syndrome differentiation of the six pathogenic factors in traditional Chinese medicine is a syndrome differentiation method that takes the six exogenous pathogenic evils as the core and analyzes the occurrence, development, and clinical manifestations of diseases.

[0003] Although the traditional method of syndrome differentiation in traditional Chinese medicine combines the four diagnostic methods of inspection, auscultation and olfaction, interrogation, and palpation, its integration method mainly relies on the subjective judgment of doctors on the external symptom characteristics of patients, lacking a comprehensive quantitative analysis of the internal physiological and pathological states of patients, resulting in incomplete feature expression and weak correlation at the data level of the syndrome differentiation model. Due to the existence of non-linear coupling between input variables and the lack of multimodal data support, the traditional syndrome differentiation model is prone to problems such as poor model convergence, uneven distribution of feature weights, and fuzzy decision boundaries in the process of feature extraction and pattern classification, thus ultimately leading to the problem of low recognition accuracy of the traditional syndrome differentiation model in practical applications.

[0004] Therefore, there is an urgent need for a method and device for intelligent identification of the six pathogenic factors in traditional Chinese medicine by integrating multimodal data. Summary of the Invention

[0005] This application provides a method and device for intelligent identification of the six pathogenic factors in traditional Chinese medicine by integrating multimodal data, which solves the problems such as poor model convergence, uneven distribution of feature weights, and fuzzy decision boundaries that are prone to occur in the process of feature extraction and pattern classification of the traditional syndrome differentiation model, thus ultimately leading to the problem of low recognition accuracy of the traditional syndrome differentiation model in practical applications.

[0006] In the first aspect of this application, a method for intelligent identification of the six pathogenic factors in traditional Chinese medicine by integrating multimodal data is provided. The method includes: obtaining multimodal data of patients with six pathogenic factor diseases in traditional Chinese medicine, where the multimodal data includes face video data, syndrome data of six pathogenic factor diseases, and sympathetic nerve activity data; through feature extraction, obtaining facial optical physiological change features based on the face video data, extracting syndrome features based on the syndrome data of six pathogenic factor diseases, and extracting sympathetic nerve activity features based on the face video data and sympathetic nerve activity data; performing feature fusion on the preprocessed facial optical physiological change features, syndrome features, and sympathetic nerve activity features; inputting the fused features into an interval judgment model established based on the KNN algorithm, and obtaining the intelligent identification result of the six pathogenic factors in traditional Chinese medicine according to the output of the interval judgment model; and outputting a corresponding syndrome differentiation and treatment plan for patients with six pathogenic factor diseases in traditional Chinese medicine based on the intelligent identification result of the six pathogenic factors in traditional Chinese medicine.

[0007] Optionally, feature extraction is performed to obtain facial optical physiological change features based on facial video data and symptom features based on the six pathogenic factors (wind, cold, summer heat, dampness, dryness, and fire) syndrome data. Specifically, this includes: performing frame decomposition and face detection operations on the facial video data to extract target region images corresponding to densely distributed blood vessels; calculating the pixel mean sequence corresponding to the R, G, and B channels in the target region image; drawing facial light change curves based on the pixel mean sequence and constructing facial optical physiological change features based on the facial light change curves; and classifying and encoding the six pathogenic factors syndrome data to output symptom features, which include the clinical manifestations corresponding to wind-induced syndrome, cold-induced syndrome, summer heat-induced syndrome, dampness-induced syndrome, dryness-induced syndrome, and fire-induced syndrome, respectively.

[0008] Optionally, sympathetic neural activity features are extracted based on facial video data and sympathetic neural activity data. Specifically, this includes: separating the pixel mean sequence FastICA signal to extract the original rPPG signal related to the heartbeat cycle; using wavelet transform and bandpass filter to remove high-frequency noise and baseline drift in the original rPPG signal to output a clean rPPG signal; calculating the RR interval sequence formed by the spacing between adjacent peaks in the clean rPPG signal, and outputting sympathetic neural activity features based on the RR interval sequence.

[0009] Optionally, the preprocessed facial optical physiological change features, symptom features, and sympathetic nerve activity features are fused. Specifically, this includes: normalizing the facial optical physiological change features, symptom features, and sympathetic nerve activity features to generate a corresponding feature information set; and inputting the feature information set into a pre-trained VGG convolutional neural network to construct fused features.

[0010] Optionally, the fusion features are input into an interval judgment model based on the KNN algorithm, and the intelligent identification result of the six pathogenic factors in traditional Chinese medicine is output according to the interval judgment model. Specifically, this includes: constructing preset fusion features based on sample data of patients with different types of the six pathogenic factors in traditional Chinese medicine, with one type of the six pathogenic factors in traditional Chinese medicine corresponding to at least one preset fusion feature. The six pathogenic factors in traditional Chinese medicine include wind-type, cold-type, summer-type, damp-type, dry-type, and fire-type; obtaining target fusion features that meet the first preset selection condition from the preset fusion features, and calculating the frequency value of each target fusion feature in different types of the six pathogenic factors in traditional Chinese medicine; sorting the frequency values, and obtaining the target types of the six pathogenic factors in traditional Chinese medicine that meet the second preset selection condition according to the sorting result; and using the target types of the six pathogenic factors in traditional Chinese medicine as the intelligent identification result of the six pathogenic factors in traditional Chinese medicine.

[0011] Optionally, obtaining the target fusion feature that meets the first preset selection condition from the preset fusion features specifically includes: calculating the Euclidean distance value between the fusion feature and multiple preset fusion features according to the interval judgment model; sorting the Euclidean distance values ​​and obtaining the target Euclidean distance that meets the first preset selection condition according to the sorting result; and taking the preset fusion feature corresponding to the target Euclidean distance as the target fusion feature.

[0012] Optionally, before outputting the corresponding syndrome differentiation and treatment plan for patients with diseases caused by the six pathogenic factors in Traditional Chinese Medicine (TCM) based on the intelligent identification results of the six pathogenic factors, a treatment database is constructed. Specifically, this includes: acquiring different types of the six pathogenic factors in TCM and multiple syndrome differentiation and treatment plans, including the wind-dispelling and exterior-releasing plan, the warming and dispersing cold plan, the summer-heat-clearing and qi-tonifying plan, the dampness-removing and spleen-strengthening plan, the dryness-moistening and yin-nourishing plan, and the heat-clearing and fire-purging plan; constructing the correspondence between the types of the six pathogenic factors in TCM and the syndrome differentiation and treatment plans, and constructing the treatment database based on the correspondence.

[0013] A second aspect of this application provides a TCM six pathogenic factors intelligent identification device that integrates multimodal data. The device includes an acquisition module, a feature fusion module, a model building module, and an intelligent identification module, wherein... The acquisition module is used to acquire multimodal data of patients with the six pathogenic factors in traditional Chinese medicine. The multimodal data includes facial video data, symptom data of the six pathogenic factors, and sympathetic nerve activity data. Through feature extraction, facial optical physiological change features are obtained based on the facial video data, symptom features are extracted based on the symptom data of the six pathogenic factors, and sympathetic nerve activity features are extracted based on the facial video data and the sympathetic nerve activity data.

[0014] The feature fusion module is used to fuse the preprocessed facial optical physiological change features, the symptom features, and the sympathetic nerve activity features.

[0015] The model building module is used to input the fused features into the interval judgment model built based on the KNN algorithm, and output the intelligent identification results of the six pathogenic factors in traditional Chinese medicine according to the interval judgment model.

[0016] The intelligent identification module is used to output the corresponding syndrome differentiation and treatment plan for patients with the six pathogenic factors in traditional Chinese medicine based on the intelligent identification results of the six pathogenic factors in traditional Chinese medicine.

[0017] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described above.

[0018] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, which is executed by a processor according to any of the methods described above.

[0019] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By quantifying differences in facial optical physiological changes, heart rhythm, and sympathetic nerve activity, the objectivity and repeatability of the diagnostic process are significantly improved. By establishing an interval judgment model between multimodal features, automatic identification and accurate classification of the six pathogenic factors are achieved. Combined with the intelligent identification results of the six pathogenic factors in traditional Chinese medicine output by the model, individualized diagnostic and treatment plans can be further matched, which can effectively reduce the bias in the manual diagnostic process, realize the transformation from experience-based diagnosis to data-driven diagnosis, thereby improving the learning ability of nonlinear correlation between multimodal features, enabling the model to achieve high-resolution discrimination of the six pathogenic factors in complex physiological feature space, and significantly improving the overall accuracy and reliability of traditional Chinese medicine diagnostic identification.

[0020] 2. The pixel mean sequence FastICA signal is separated to extract the original rPPG signal related to the cardiac cycle. Wavelet transform and bandpass filter are used to remove high-frequency noise and baseline drift from the original rPPG signal to output a pure rPPG signal. The RR interval sequence formed by the distance between adjacent peaks in the pure rPPG signal is calculated, and the sympathetic nerve activity characteristics are output based on the RR interval sequence. This enables refined identification of the dynamic regulation state of the autonomic nervous system at the cardiac cycle level, and can simultaneously reflect the balance between the sympathetic and parasympathetic nervous systems and their fluctuation patterns under different TCM pathological types. Through this feature extraction and parameter analysis process, the excitation or inhibition state of the patient's autonomic nervous function can be assessed in real time under non-invasive conditions, providing objective quantitative evidence for identifying the influence of external pathogens such as cold, heat, dampness, dryness, wind, and fire on the body's nervous regulation system during the process of pathogenic factors.

[0021] 3. Based on the interval judgment model, calculate the Euclidean distance between the fusion feature and multiple preset fusion features respectively; sort the Euclidean distance values, and obtain the target Euclidean distance that meets the first preset selection condition based on the sorting result; take the preset fusion feature corresponding to the target Euclidean distance as the target fusion feature, and thus achieve accurate discrimination of different types of traditional Chinese medicine six pathogenic factors by quantitatively comparing the distance between the test sample and the preset fusion features of each category in the high-dimensional feature space. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a method for intelligent identification of the six pathogenic factors in Traditional Chinese Medicine that integrates multimodal data, as provided in an embodiment of this application. Figure 2This is a schematic diagram of a module of a TCM six pathogenic factors intelligent identification device that integrates multimodal data, provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0023] Explanation of reference numerals in the attached figures: 21. Acquisition module; 22. Feature fusion module; 23. Model building module; 24. Intelligent recognition module; 301. Processor; 302. Communication bus; 303. User interface; 304. Network interface; 305. Memory. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0025] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0026] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0027] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0028] Please refer to Figure 1 The flowchart illustrates a method for intelligent identification of the six pathogenic factors in traditional Chinese medicine that integrates multimodal data, provided in an embodiment of this application. The flowchart mainly includes the following steps: S101 to S105.

[0029] Step S101: Obtain multimodal data of patients with the six pathogenic factors in traditional Chinese medicine. The multimodal data includes facial video data, symptoms of the six pathogenic factors, and sympathetic nerve activity data.

[0030] Specifically, the acquisition steps for facial video data are as follows: A certain number of facial video data points from patients with the six pathogenic factors (TCM-related diseases) are collected using a camera device. A standard webcam is placed 0.5m directly in front of the patient's face, or it can be mounted above a computer monitor. The camera is connected to the computer via USB. The operator uses Matlab software to control the camera to capture video, ensuring the face is completely within the video frame. All videos are continuously captured for 10 seconds at a resolution of 640×480, a frame rate of 30 frames per second, and in RGB color space, and saved in AVI format.

[0031] The data collection steps for the syndrome data of the six pathogenic factors in Traditional Chinese Medicine (TCM) are as follows: Information on the syndromes of the six pathogenic factors in TCM is obtained from the patient, and a set of characteristic information on the syndromes of the six pathogenic factors in TCM is established. For example, wind-induced syndrome refers to the invasion of wind pathogens into the skin and meridians, leading to dysfunction of the body's defensive functions and manifesting symptoms consistent with the characteristics of "wind." In addition, pathogenic factors such as cold, heat, fire, dampness, phlegm, water, and toxins often attach to wind and invade the body, forming different concurrent syndromes, such as wind-cold syndrome, wind-heat syndrome, wind-fire syndrome, wind-dampness syndrome, wind-phlegm syndrome, wind-water syndrome, and wind-toxin syndrome. The symptoms of wind-induced syndrome generally include: aversion to wind, slight fever, sweating, thin white tongue coating, and floating and slow pulse; or nasal congestion, runny nose, sneezing, or accompanied by itchy and sore throat, cough; or sudden onset of wheals, itchy skin, rashes; or sudden onset of numbness of the skin, facial paralysis; or muscle stiffness, spasms, convulsions; or migratory pain in the limbs and joints; or new onset of facial and limb edema, etc. The key points for diagnosis are: aversion to wind, slight fever, sweating, and a floating and slow pulse; or sudden onset of wheals, itching, numbness, migratory pain in the limbs and joints, and facial edema, etc.

[0032] The data acquisition steps for sympathetic nerve activity are as follows: To achieve non-contact acquisition of sympathetic nerve activity signals, the principle of visible light remote optical volumetric recording (RPPG) is used, combined with the optical reflection characteristics of the patient's facial video signal. The acquisition environment must be under stable lighting, with illuminance maintained between 300 and 500 lx, avoiding direct light and flickering light sources. The subject should maintain a natural sitting posture, with their head facing the camera device, keeping their face unobstructed, expression relaxed, and avoiding blinking or head movement in a resting state. The camera device should be an RGB color video acquisition device with a frame rate of no less than 30 fps and a resolution of no less than 640×480, installed approximately 0.5m away from the patient, ensuring that the camera's optical axis is perpendicular to the center line of the face, and the camera angle error does not exceed 10°. The camera device is connected to a computer via USB or a wired interface. The acquisition end uses a Matlab or Python control module for video acquisition and time synchronization marking; during the acquisition process, a timestamp sequence is automatically generated and embedded in the video frame file for subsequent synchronization processing. To ensure signal integrity, environmental self-checks and white balance calibrations must be performed before acquisition to control the average grayscale value difference of the R, G, and B channels within ±10. Patients should maintain quiet breathing during acquisition to avoid interference from sympathetic excitation responses. The video acquisition duration is set to 10-15 seconds. After acquisition, the video is automatically saved in AVI or MP4 format, with the patient number, acquisition date, acquisition device number, ambient illuminance, and body position identifier written in the file header. To assist in the accurate localization of sympathetic nerve activity data, ambient light intensity sensor output, ECG reference signals, and skin temperature change signals can be acquired simultaneously for subsequent signal comparison and verification. After acquisition, the video file undergoes integrity verification, removing dropped frames or abnormally exposed segments to generate a stable video frame sequence dataset. This video frame sequence dataset, as the original form of sympathetic nerve activity data, contains facial optical reflection information and time series identifiers, providing input for subsequent optical signal processing and sympathetic nerve activity state analysis.

[0033] Step S102: Through feature extraction, facial optical physiological change features are obtained based on facial video data, symptom features are extracted based on the six pathogenic factors disease symptom data, and sympathetic nerve activity features are extracted based on facial video data and sympathetic nerve activity data.

[0034] Specifically, using facial video data as input, the system extracts regions of interest (ROIs) with dense blood vessel distribution through frame decomposition, face detection, and region selection. It then calculates the R, G, and B channel pixel mean sequences and plots facial light variation curves to characterize the facial optical and physiological changes caused by blood flow and cardiac activity. Secondly, the system performs structured encoding on the symptom data of the six pathogenic factors (wind, cold, summer heat, dampness, dryness, and fire), transforming the corresponding symptoms, pulse, and tongue information into multi-dimensional vectors to extract symptom features. Finally, based on facial video data and sympathetic nervous system activity data, the system obtains a set of feature parameters reflecting the activity level of the autonomic nervous system through signal synchronization, temporal alignment, and cardiac cycle recognition. These parameters include indicators related to sympathetic excitation and parasympathetic regulation.

[0035] In one possible implementation, step S102 further includes: specifically, performing frame decomposition and face detection operations on the face video data to extract the target region image corresponding to the densely distributed blood vessel area; calculating the pixel mean sequence corresponding to the R, G, and B channels in the target region image respectively; drawing a facial light change curve based on the pixel mean sequence, and constructing facial optical physiological change features based on the facial light change curve; classifying and encoding the syndrome data of the six pathogenic factors to output syndrome features, including the clinical manifestation features corresponding to wind-induced syndrome, cold-induced syndrome, summer-induced syndrome, dampness-induced syndrome, dryness-induced syndrome, and fire-induced syndrome respectively.

[0036] Specifically, using the pixel mean sequence output from the previous stage in step S102 as input, independent component analysis is first performed to separate the heartbeat-related source signals. The three-channel observation vector after zero-meaning and whitening is then set as follows: Independent source vectors are The non-singular mixture matrix is The unmixing matrix is Its basic model is:

[0037] in, This represents the three-channel pixel mean observation vector arranged in time. Let represent the vector of statistically independent source components to be determined; This represents the blending matrix caused by both scene and skin optical reflections; Let represent the unmixing matrix, and the objective of solving it is to maximize the non-Gaussianity of the components; This indicates the time index corresponding to the frame.

[0038] To solve Using FastICA fixed-point iteration, the weight vector of a single independent component is denoted as... The derivative of the comparison function is The one-step update after second-order moment whitening is:

[0039] in, or ; To compare the slope parameter of the function, its value range is generally within... ; This represents the expectation for the sample. Denotes the Euclidean norm; the iterative convergence threshold is set to Used for determination Whether the difference between adjacent iterations is small enough.

[0040] The component with the highest correlation coefficient to the mean sequence of green channel pixels among the several source components is denoted as the original rPPG waveform. Subsequently, wavelet denoising and bandpass filtering are cascaded to obtain a clean rPPG signal. Let... The discrete wavelet decomposition coefficients are Soft threshold denoising uses:

[0041] in, Indicates the first Layer Wavelet coefficients; This represents the coefficients after thresholding; For the first The layer threshold can be an adaptive layer threshold that estimates the proportion of the robust median, and the range is selected according to the noise level. Represents a symbolic function.

[0042] Signal after thresholding reconstruction The signal is passed through a bandpass filter to preserve the dominant heart rate and harmonic components, making the passband [value missing]. Then the transfer function of the ideal bandpass in the frequency domain is:

[0043] in, Indicates frequency; and These are the lower and upper cutoff frequencies, which can be set according to the resting pulse rate range of the subject population, for example... , Selected within the actual value range; This is the frequency domain gain function. In practical implementations, this ideal response is approximated using equivalent IIR or FIR designs. The filtered output is defined as the pure rPPG signal. .

[0044] by To perform peak detection to locate the heartbeat arrival time, let the detected peak be... The effective peak time is The interval between adjacent peaks is defined as the RR interval:

[0045] in, Indicates the peak positions sorted in ascending order of time; This indicates the time interval between two consecutive heartbeats; to ensure robustness, it can be adjusted... The sequence performs spurious peak removal and minimum peak spacing constraint. The minimum peak spacing parameter value is consistent with the previous bandpass passband to avoid false detection of octaves and noise.

[0046] based on To calculate the feature set of sympathetic neural activity, time-domain representative indices include SDNN and RMSSD, which are defined as follows:

[0047]

[0048] in, This indicates the number of valid RR interval samples; Represents the arithmetic mean of the RR interval samples; SDNN measures the overall level of variability; RMSSD is sensitive to the difference between adjacent heartbeats.

[0049] Frequency domain representativeness indexes are estimated using the Welch method to measure the power spectral density. Then, by integrating within the specified frequency band, LF and HF are obtained, along with their ratio LF / HF, which is defined as:

[0050] in, LF represents the power spectral density of the RR interval sequence; LF reflects the low-frequency power component modulated by both sympathetic and parasympathetic induction; HF reflects the high-frequency component dominated by parasympathetic induction; LF / HF represents the balance relationship between sympathetic and parasympathetic induction.

[0051] The aforementioned time-domain and frequency-domain indicators, along with necessary nonlinear indicators, such as those based on the Poincaré diagram, are combined. By merging these components, a characteristic vector of sympathetic nerve activity can be constructed. This is the final output of this embodiment; in the processing chain of this embodiment, the input is a pixel mean sequence, which is obtained through independent component extraction. After wavelet and bandpass processing, the following was obtained: Peak values ​​were obtained and The final calculation yielded .

[0052] Step S103: The preprocessed facial optical physiological changes, symptom features, and sympathetic nerve activity features are fused together.

[0053] Specifically, facial photophysiological changes, symptom features, and sympathetic neural activity features are normalized to maintain consistent distribution within the same numerical range. Common normalization methods include Z-score standardization or min-max linear transformation. Secondly, multimodal features are dimensionally aligned and temporally matched. A feature alignment matrix is ​​used to achieve temporal synchronization and dimensional mapping between samples, combining continuous signal features and discrete symptom features into a unified fusion feature vector. Then, a multimodal fusion algorithm based on convolutional neural networks is employed to map features from different sources to the latent space feature layer, and cross-modal correlation features are extracted using convolutional kernels. The selection of convolutional kernel size, stride, and pooling method in the network structure prioritizes minimizing information loss. Finally, the fused feature set is output through a fully connected layer, achieving multi-layer semantic coupling of facial photophysiological changes, symptom manifestations, and sympathetic neural activity features, providing a unified multimodal fusion feature representation for subsequent interval judgment models.

[0054] In one possible implementation, step S103 further includes: normalizing the facial optical physiological change features, symptom features, and sympathetic nerve activity features to generate a corresponding feature information set; and inputting the feature information set into a pre-trained VGG convolutional neural network to construct fused features.

[0055] Specifically, facial optophysiological changes, symptom features, and sympathetic activity features are normalized separately. Normalization aims to eliminate dimensional differences and scale inconsistencies between different modalities, ensuring comparability of features within the same numerical range. For example, optophysiological changes are typically continuous time series, while symptom features are mostly discrete classification codes, and sympathetic activity features originate from physiological parameters such as heart rate variability. After standardization, each feature is adjusted to the same range in the numerical domain, thus preventing uneven feature distribution from causing bias in the model during training.

[0056] The normalized features are then combined into a multi-dimensional feature information set. This set, with samples as rows and features as columns, forms a unified multimodal feature matrix, which serves as the input for the deep convolutional feature extraction stage. During this process, the fused input vector for each sample includes temporal segment features from optical physiological signals, symptom structure coding features, and sympathetic neural modulation signal features.

[0057] Subsequently, the feature information set is input into a pre-trained VGG convolutional neural network. This network has a typical layered structure, including multiple convolutional layers, pooling layers, and several fully connected layers. The convolutional layers are used to extract local feature associations, performing feature convolution on the input matrix through sliding convolution kernels to obtain cross-modal association features, thereby capturing the implicit mapping relationship between facial signal fluctuations and TCM symptom manifestations; the pooling layers are used for feature downsampling, reducing feature dimensionality and suppressing noise interference, making the model robust to local perturbations; the activation layers introduce nonlinear transformations, enabling the model to learn complex intermodal coupling features; the fully connected layers perform high-dimensional feature combination on the convolutional outputs to form a global representation.

[0058] During feature mapping, the low-level convolutional layers of the VGG network focus on capturing variations in the texture and color distribution of the underlying image, mapping these variations to subtle intensity fluctuations in facial optical physiological signals. Mid-level convolutional layers extract mesoscale patterns related to changes in sympathetic neural activity, such as the coupling features between facial blood flow rhythm and neural excitation. High-level convolutional layers learn abstract semantic representations to identify the distribution patterns of different types of features in the overall modal space. Through backpropagation optimization, the convolutional kernel parameters are updated to minimize feature reconstruction errors, ensuring that the fused features maintain structural consistency and discriminative power across different modalities.

[0059] Finally, the high-level feature maps output by the VGG network are flattened and nonlinearly mapped to generate a fused feature vector. This fused feature vector integrates the semantic and statistical correlations among facial optical change patterns, sympathetic nerve activity rhythms, and TCM symptom codes, forming a high-dimensional representation for the interval judgment model input. This fusion mechanism achieves a deep abstract expression from raw multimodal data to a unified feature space, providing a complete and hierarchical fused feature foundation for subsequent intelligent identification of the six pathogenic factors in TCM.

[0060] Step S104: Input the fused features into the interval judgment model established based on the KNN algorithm, and output the intelligent identification results of the six pathogenic factors in traditional Chinese medicine based on the interval judgment model.

[0061] Specifically, for the feature sample set established during the training phase, each of the six pathogenic factors in Traditional Chinese Medicine (TCM) corresponds to several fused feature samples, with each sample class constituting a specific feature interval. Next, for the sample to be tested, the distance between its fused feature vector and the sample sets of each class is calculated, and the similarity to each of the six pathogenic factors is determined using Euclidean distance or cosine similarity metric. Then, the closest samples are selected according to distance ranking, and the distribution frequency of the category to which these samples belong within the six pathogenic factors is statistically analyzed to determine the probability of the target type appearing. Finally, the type with the highest frequency or the largest overall weight is output as the intelligent identification result. This step enables the model to determine the intervals between different types of the six pathogenic factors based on multimodal fused features, thereby outputting structured and interpretable diagnostic classification results, providing a basis for subsequent diagnosis and treatment.

[0062] In one possible implementation, step S104 further includes: specifically, constructing preset fusion features based on sample data of patients with different types of TCM six pathogenic factors, where each TCM six pathogenic factor type corresponds to at least one preset fusion feature, and the TCM six pathogenic factors types include wind-type, cold-type, summer-type, damp-type, dry-type, and fire-type; obtaining target fusion features among the preset fusion features that meet the first preset selection condition, and calculating the frequency value of each target fusion feature in different TCM six pathogenic factors; calculating the Euclidean distance values ​​between the fusion features and multiple preset fusion features according to the interval judgment model; sorting the Euclidean distance values, and obtaining the target Euclidean distance that meets the first preset selection condition according to the sorting result; using the preset fusion feature corresponding to the target Euclidean distance as the target fusion feature; sorting the frequency values, and obtaining the target TCM six pathogenic factors that meet the second preset selection condition according to the sorting result; and using the target TCM six pathogenic factors as the TCM six pathogenic factor intelligent identification result.

[0063] Specifically, a pre-defined set of fusion features is constructed using sample data from patients with different TCM six pathogenic factors types. Then, target fusion feature selection and target TCM six pathogenic factors type determination are performed under an interval judgment model. Let the fusion feature vector be... The preset fusion feature set is ,in This represents the preset fusion features derived from the sample data. This indicates the corresponding TCM six pathogenic factors type label. To measure similarity, Euclidean distance is used for pairing. Measured with each preset fusion feature:

[0064] in, This represents the fusion features of the sample to be judged; Indicates the first One preset fusion feature; Represents the Euclidean distance value; This represents the L2 norm. This metric reflects the proximity of fused features in a high-dimensional space using geometric distance.

[0065] Under the first preset screening condition, target Euclidean distances that satisfy the ranking and threshold are selected from the entire Euclidean distance value sequence, and a subset of target fusion features is obtained accordingly. Let the ranking threshold be... Distance threshold is ,make Indicates by The target fusion feature subset is defined as follows, ranked from smallest to largest:

[0066] in, This represents the set of target fusion features that meet the first preset screening criteria; This represents the threshold for the number of nearest neighbors, and its value is a positive integer. It is used to limit the number of the nearest samples involved in the judgment. This represents the distance threshold, used to remove abnormal nearest neighbors that are ranked high but whose absolute distance is too large. This indicates a logical AND operation. The condition employs a dual-constraint strategy of ranking constraint + distance constraint to ensure that the target fused features are both close to... However, it does not exceed the reasonable similarity boundary.

[0067] The frequency values ​​of the target fusion feature set in different types of the six pathogenic factors in Traditional Chinese Medicine are calculated to form the input of the second preset screening condition. Let the type set be wind-type, cold-type, summer-type, damp-type, dry-type, and fire-type. Then the frequency count and the relative frequency are defined as:

[0068] in, Indicates the type of target fusion features. The number of samples; Representation type The frequency of occurrence; The cardinality representing the target fusion features; This is an indicator function. The frequency value is used to measure the dominance of different types of the six pathogenic factors in traditional Chinese medicine within the nearest neighbor subset.

[0069] To improve the stability of type determination in scenarios with non-uniform nearest neighbor distribution, distance-weighted voting is introduced in the second preset screening condition. Target fusion features are weighted according to similarity and then comprehensively ranked. The target type is determined by the highest weighted score.

[0070] in, Indicates the first The distance weights of the fused features of each target are determined, with larger weights for smaller distances. To prevent the stabilization constant from having a denominator of zero, it is taken as a very small positive number; For type The weighted score; The target is one of the six pathogenic factors in Traditional Chinese Medicine. In implementation, the second preset screening condition can be defined as... Maximum or Exceeding the minimum confidence threshold It ranks first among all types to avoid uncertain judgments caused by low confidence situations.

[0071] The target type of the six pathogenic factors in Traditional Chinese Medicine (TCM) is output as the intelligent identification result of the six pathogenic factors in TCM. The target fusion features used in this determination and their corresponding Euclidean distance values ​​are retained to support subsequent online optimization or consistency verification. In the above processing chain, the input is the fusion features output in step S103. The target fusion feature set is obtained by calculating the Euclidean distance value and applying it to the first preset screening condition. The target TCM six pathogenic factors types were obtained by calculating the frequency value and weighted score and applying the second preset screening criteria. The output of the preceding steps is always used as the input for the subsequent steps, ensuring consistency in terminology and data inheritance.

[0072] in, Dimensions To fuse the dimension of the feature space, the feature mapping design is derived from step S103; The total amount of preset fusion features is constructed from historical sample data; The recommended value can be selected based on the principle of minimizing the validation set error. The suggested range is consistent with the feature scale, often using the first feature on the validation set. The median distance or quantile setting of nearest neighbors; Recommendation Positive numbers within the range to avoid division by zero and control over-amplification effects; If confidence thresholds are enabled, they can be set according to the application's fault tolerance requirements. Internal selection is used to reject the application or trigger a review process.

[0073] Step S105: Based on the results of the intelligent identification of the six pathogenic factors in traditional Chinese medicine, output the corresponding syndrome differentiation and treatment plan for patients with diseases caused by the six pathogenic factors in traditional Chinese medicine.

[0074] Specifically, the intelligent identification results of the six pathogenic factors in Traditional Chinese Medicine (TCM) are used to classify patients into different syndrome types and match corresponding treatment plans, achieving a closed-loop process from automatic identification to individualized intervention. This step maps the identification results to a pre-established treatment database through the intelligent identification module. The database stores standardized syndrome differentiation and treatment plans corresponding to different types of the six pathogenic factors, including the main treatment principles, core prescription combinations, treatment order, and conditioning cycle.

[0075] Once the identification results are determined, the system first retrieves the corresponding type of the six pathogenic factors and calls its basic treatment template; then, it adjusts the parameters based on the individual characteristics of the patient to achieve personalized fine-tuning of the treatment plan. For example, for patients with wind-type pathogenic factors who have high sympathetic nerve excitability, the system prioritizes recommending a conditioning plan that dispels wind, calms the liver, and relieves nerve stress; for patients with cold-type pathogenic factors who exhibit reflexive contraction of peripheral blood flow, the system outputs a treatment plan that is more inclined to warm the yang, dispel cold, and improve circulation.

[0076] In the final output generation stage, the module packages the diagnosis results, treatment parameters, and corresponding treatment suggestions into a standard output format, including treatment goals, recommended prescriptions, auxiliary conditioning suggestions, and treatment confidence labels, thereby completing the automatic generation and output of TCM six pathogenic factors intelligent identification results into a diagnosis and treatment plan.

[0077] In one possible implementation, step S105 further includes: acquiring different types of the six pathogenic factors in traditional Chinese medicine and multiple syndrome differentiation and treatment plans, including the wind-dispelling and exterior-releasing plan, the warming and cold-dispersing plan, the summer-heat-clearing and qi-tonifying plan, the dampness-removing and spleen-strengthening plan, the dryness-moistening and yin-nourishing plan, and the heat-clearing and fire-purging plan; constructing the correspondence between the types of the six pathogenic factors in traditional Chinese medicine and the syndrome differentiation and treatment plans, and constructing a treatment database based on the correspondence.

[0078] Specifically, in this implementation, the syndrome differentiation and treatment plans derived from classical prescriptions, clinical pathways, and medical case data are first summarized and organized. The plan types are limited to wind-dispelling and exterior-releasing plans, warming yang and dispersing cold plans, clearing summer heat and replenishing qi plans, dampness-dispelling and spleen-strengthening plans, dryness-moistening and yin-nourishing plans, and heat-clearing and fire-purging plans. Standardized entries are established for each plan. Each entry should include at least the treatment goal, key points of treatment, core prescriptions and modification ideas, keywords of appropriate syndromes, contraindications and conditions for caution, intervention cycle and re-evaluation time, and follow-up and adverse reaction record fields. Through consistency verification, duplicate and contradictory entries are eliminated to form a reusable plan library draft to support the mapping construction from different types of the six pathogenic factors in traditional Chinese medicine to treatment plans.

[0079] Based on the six types of pathogenic factors in Traditional Chinese Medicine (TCM) – wind-induced, cold-induced, summer-induced, damp-induced, dry-induced, and fire-induced – a set of type definitions and a set of syndrome anchor points are established. The primary symptoms, secondary symptoms, and key points of tongue and pulse for each type are extracted as standardized labels, which are then bidirectionally matched with the "matching syndrome keywords" field in the aforementioned draft treatment plan library. Priority is given to establishing a one-to-one primary correspondence. In cases where there are overlapping or unclear boundaries, secondary correspondences are introduced and assigned priority indicators. Finally, a correspondence table of "TCM Six Pathogenic Factor Types - Syndrome Differentiation and Treatment Plans" is formed, which serves as the core index of the treatment database.

[0080] When performing engineered database storage of the corresponding relationship table, a structured data model is used to store three layers of information: type definition, scheme entries, and mapping index. Each mapping record is configured with version number, effective range, reviewer, and data source metadata. Integrity constraints are set to ensure that each of the six pathogenic factors in traditional Chinese medicine maps to at least one syndrome differentiation and treatment scheme. Each scheme entry must have two mandatory fields: treatment principle and core prescription. Empty mapping and circular mapping are prohibited to ensure that the database can be stably retrieved and interpreted by the intelligent identification module.

[0081] Before going live, consistency and robustness verification are performed. The "type-scheme" mapping is played back offline using historical dialectical samples to verify the consistency rate between the scheme call results and manual interpretation. For mismatched samples, the distribution of their syndrome anchor points is analyzed and the keyword descriptions are optimized. If necessary, the priority and trigger threshold of the secondary correspondence relationship are configured for the mixed scenarios, thereby reducing the scheme uncertainty of boundary samples and improving interpretability.

[0082] During the operation phase, when step S104 outputs the results of the intelligent identification of the six pathogenic factors in traditional Chinese medicine, the intelligent identification module accesses the treatment database using the result type as the query key. First, it retrieves the standardized syndrome differentiation and treatment plan that matches the main correspondence. Then, based on the patient's individualized information, including the weight of sympathetic features, the strength of sympathetic nerve activity, and the stability of facial optical physiological changes, it fine-tunes the addition and subtraction ideas, treatment order, and review time points in the plan items, generating an individualized plan that can be directly used for medical order preparation. The call records, parameter values, and plan version number are archived together to support traceability and continuous optimization.

[0083] To ensure continuous evolution, a database maintenance process and a gray-scale release mechanism have been established. New or revised treatment plans must undergo item completeness checks, mapping conflict detection, and small-sample safety assessments before they can take effect. Mismatched and low-confidence cases found in clinical feedback are periodically collected, labeled, and reviewed. The corresponding relationship table is updated in batches, while the old version is retained for regression comparison. This allows the treatment database to gradually absorb new evidence, cover new concomitant situations, and form a closed-loop improvement with the intelligent identification model while ensuring stability.

[0084] Please refer to Figure 2 This document illustrates a schematic diagram of the modules of a TCM six pathogenic factors intelligent identification device that integrates multimodal data, according to an embodiment of this application. The device includes an acquisition module 21, a feature fusion module 22, a model building module 23, and an intelligent identification module 24. The acquisition module 21 is used to acquire multimodal data of patients with the six pathogenic factors in traditional Chinese medicine. The multimodal data includes facial video data, symptom data of the six pathogenic factors, and sympathetic nerve activity data. Through feature extraction, facial optical physiological change features are acquired based on the facial video data, symptom features are extracted based on the symptom data of the six pathogenic factors, and sympathetic nerve activity features are extracted based on the facial video data and the sympathetic nerve activity data.

[0085] The feature fusion module 22 is used to fuse the preprocessed facial optical physiological change features, the symptom features, and the sympathetic nerve activity features.

[0086] The model building module 23 is used to input the fused features into the interval judgment model built based on the KNN algorithm, and output the intelligent identification results of the six pathogenic factors in traditional Chinese medicine according to the interval judgment model.

[0087] The intelligent identification module 24 is used to output the syndrome differentiation and treatment plan corresponding to the patients with the six pathogenic factors in traditional Chinese medicine based on the intelligent identification results of the six pathogenic factors in traditional Chinese medicine.

[0088] In one possible implementation, the acquisition module 21 is used to acquire facial optical physiological change features based on facial video data through feature extraction, and to extract symptom features based on the six pathogenic factors disease symptom data. Specifically, this includes: performing frame decomposition and face detection operations on the facial video data to extract the target region image corresponding to the densely distributed blood vessel area; calculating the pixel mean sequence corresponding to the R, G, and B channels in the target region image respectively; drawing a facial light change curve based on the pixel mean sequence, and constructing facial optical physiological change features based on the facial light change curve; classifying and encoding the six pathogenic factors disease symptom data to output symptom features, which include the clinical manifestation features corresponding to wind pathogenic factors syndrome, cold pathogenic factors syndrome, summer heat pathogenic factors syndrome, dampness pathogenic factors syndrome, dryness pathogenic factors syndrome, and fire pathogenic factors syndrome respectively.

[0089] In one possible implementation, the acquisition module 21 is used to extract sympathetic neural activity features based on face video data and sympathetic neural activity data, specifically including: separating the pixel mean sequence FastICA signal to extract the original rPPG signal related to the heartbeat cycle; using wavelet transform and bandpass filter to remove high-frequency noise and baseline drift in the original rPPG signal to output a pure rPPG signal; calculating the RR interval sequence formed by the spacing between adjacent peaks in the pure rPPG signal, and outputting sympathetic neural activity features based on the RR interval sequence.

[0090] In one possible implementation, the feature fusion module 22 is used to fuse the preprocessed facial optical physiological change features, symptom features, and sympathetic nerve activity features. Specifically, it includes: normalizing the facial optical physiological change features, symptom features, and sympathetic nerve activity features to generate a corresponding feature information set; and inputting the feature information set into a pre-trained VGG convolutional neural network to construct fused features.

[0091] In one possible implementation, the model building module 23 is used to input the fusion features into an interval judgment model built based on the KNN algorithm, and output the TCM six pathogenic factors intelligent identification result according to the interval judgment model. Specifically, it includes: constructing preset fusion features based on sample data of patients with different TCM six pathogenic factors, where each TCM six pathogenic factor type corresponds to at least one preset fusion feature. The TCM six pathogenic factors types include wind-type, cold-type, summer-type, damp-type, dry-type, and fire-type; obtaining target fusion features that meet the first preset selection condition from the preset fusion features, and calculating the frequency value of each target fusion feature in different TCM six pathogenic factors types; sorting the frequency values, and obtaining the target TCM six pathogenic factors that meet the second preset selection condition according to the sorting result; and using the target TCM six pathogenic factors as the TCM six pathogenic factors intelligent identification result.

[0092] In one possible implementation, the model building module 23 is used to obtain target fusion features that meet the first preset selection conditions from the preset fusion features, specifically including: calculating the Euclidean distance values ​​between the fusion features and multiple preset fusion features according to the interval judgment model; sorting the Euclidean distance values, and obtaining the target Euclidean distance that meets the first preset selection conditions according to the sorting result; and using the preset fusion feature corresponding to the target Euclidean distance as the target fusion feature.

[0093] In one possible implementation, the intelligent identification module 24 is used to construct a treatment database before outputting the corresponding syndrome differentiation and treatment plan for patients with diseases caused by the six pathogenic factors in traditional Chinese medicine based on the intelligent identification results of the six pathogenic factors. Specifically, this includes: acquiring different types of the six pathogenic factors in traditional Chinese medicine and multiple syndrome differentiation and treatment plans, including the wind-dispelling and exterior-releasing plan, the warming and dispersing cold plan, the summer-heat-clearing and qi-tonifying plan, the dampness-removing and spleen-strengthening plan, the dryness-moistening and yin-nourishing plan, and the heat-clearing and fire-purging plan; constructing the correspondence between the types of the six pathogenic factors in traditional Chinese medicine and the syndrome differentiation and treatment plans, and constructing the treatment database based on the correspondence.

[0094] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0095] This application also provides an electronic device. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: at least one processor 301, at least one communication bus 302, a user interface 303, at least one network interface 304, and a memory 305.

[0096] The communication bus 302 is used to enable communication between these components.

[0097] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0098] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0099] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0100] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. (Refer to...) Figure 3 The memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a TCM six pathogenic factors intelligent identification application that integrates multimodal data.

[0101] exist Figure 3In the illustrated electronic device, the user interface 303 is primarily used to provide an input interface for the user and acquire user input data; while the processor 301 can be used to call the TCM Six Evils Intelligent Identification Application stored in the memory 305, which stores fused multimodal data. When executed by one or more processors 301, the electronic device performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0102] This application also provides a computer-readable storage medium storing instructions. When executed by one or more processors, these instructions cause an electronic device to perform one or more of the methods described in the above embodiments.

[0103] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0104] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0108] The above description is merely an exemplary embodiment disclosed in this application and should not be construed as limiting the scope of this application. Any equivalent changes and modifications made in accordance with the teachings of this application shall still fall within the scope of this application.

[0109] This application is intended to cover any variations, uses, or adaptations disclosed herein that follow the general principles disclosed herein and include common knowledge or customary technical means in the art that are not described in this application.

Claims

1. A method for intelligent identification of the six pathogenic factors in Traditional Chinese Medicine by integrating multimodal data, characterized in that, The method includes: Acquire multimodal data of patients with diseases caused by the six pathogenic factors in Traditional Chinese Medicine, including facial video data, symptom data of the six pathogenic factors, and sympathetic nerve activity data; Through feature extraction, facial optical physiological change features are obtained based on the facial video data, symptom features are extracted based on the six pathogenic factors disease symptom data, and sympathetic nerve activity features are extracted based on the facial video data and the sympathetic nerve activity data. The preprocessed facial photophysiological changes, symptom features, and sympathetic nerve activity features are fused together. The fused features are input into the interval judgment model established based on the KNN algorithm, and the intelligent identification results of the six pathogenic factors in traditional Chinese medicine are output according to the interval judgment model. Based on the results of the intelligent identification of the six pathogenic factors in Traditional Chinese Medicine, the corresponding syndrome differentiation and treatment plan for patients with the diseases caused by the six pathogenic factors in Traditional Chinese Medicine is output.

2. The method according to claim 1, characterized in that, The step of extracting facial optical and physiological change features based on the facial video data and extracting symptom features based on the six pathogenic factors disease symptom data specifically includes: Frame decomposition and face detection operations are performed on the face video data to extract the target region image corresponding to the densely distributed blood vessel area; Calculate the pixel mean sequence corresponding to the R, G, and B channels in the target region image respectively; A facial light variation curve is plotted based on the pixel mean sequence, and the facial optical physiological variation features are constructed based on the facial light variation curve. The symptom data of the six pathogenic factors are classified and coded to output the symptom features, which include the clinical manifestations corresponding to wind-induced syndrome, cold-induced syndrome, summer-induced syndrome, dampness-induced syndrome, dryness-induced syndrome and fire-induced syndrome respectively.

3. The method according to claim 2, characterized in that, The extraction of sympathetic nerve activity features based on the facial video data and the sympathetic nerve activity data specifically includes: The pixel mean sequence FastICA signal is separated to extract the raw rPPG signal related to the heart cycle; Wavelet transform and bandpass filter are used to remove high-frequency noise and baseline drift from the original rPPG signal to output a clean rPPG signal; Calculate the RR interval sequence formed by the spacing between adjacent peaks in the pure rPPG signal, and output the sympathetic nerve activity characteristics based on the RR interval sequence.

4. The method according to claim 1, characterized in that, The feature fusion of the preprocessed facial optical physiological changes, symptom features, and sympathetic nerve activity features specifically includes: The facial optical physiological changes, the symptom features, and the sympathetic nerve activity features are normalized to generate a corresponding set of feature information. The set of feature information is input into a pre-trained VGG convolutional neural network to construct the fused features.

5. The method according to claim 1, characterized in that, The process of inputting fused features into an interval judgment model based on the KNN algorithm and outputting the intelligent identification results of the six pathogenic factors in traditional Chinese medicine based on the interval judgment model specifically includes: Preset fusion features are constructed based on sample data of patients with different types of the six pathogenic factors in traditional Chinese medicine. Each of the six pathogenic factors in traditional Chinese medicine corresponds to at least one of the preset fusion features. The six pathogenic factors in traditional Chinese medicine include wind-induced, cold-induced, summer-induced, damp-induced, dryness-induced, and fire-induced types. Obtain the target fusion features that satisfy the first preset selection conditions from the preset fusion features, and calculate the frequency value of each target fusion feature in different types of the six pathogenic factors in traditional Chinese medicine; The frequency values ​​are sorted, and based on the sorting results, the target TCM six pathogenic factors that meet the second preset selection conditions are obtained. The target type of the six pathogenic factors in traditional Chinese medicine is used as the result of the intelligent identification of the six pathogenic factors in traditional Chinese medicine.

6. The method according to claim 5, characterized in that, The acquisition of the target fusion feature that satisfies the first preset selection condition from the preset fusion features specifically includes: Based on the interval judgment model, calculate the Euclidean distance between the fused feature and multiple preset fused features respectively; The Euclidean distance values ​​are sorted, and the target Euclidean distance that satisfies the first preset selection condition is obtained based on the sorting result. The preset fusion feature corresponding to the target Euclidean distance is used as the target fusion feature.

7. The method according to claim 5, characterized in that, Before outputting the corresponding syndrome differentiation and treatment plan for patients with the six pathogenic factors in Traditional Chinese Medicine (TCM) based on the intelligent identification results of the six pathogenic factors, a treatment database is constructed, specifically including: Obtain different types of the six pathogenic factors in traditional Chinese medicine and multiple treatment plans based on syndrome differentiation. The treatment plans include the wind-dispelling and exterior-releasing plan, the yang-warming and cold-dispersing plan, the summer-heat-clearing and qi-tonifying plan, the dampness-removing and spleen-strengthening plan, the dryness-moistening and yin-nourishing plan, and the heat-clearing and fire-purging plan. Construct a correspondence between the six pathogenic factors in traditional Chinese medicine and the corresponding treatment plans, and build the treatment database based on the correspondence.

8. A smart identification device for the six pathogenic factors in Traditional Chinese Medicine that integrates multimodal data, characterized in that, The device includes an acquisition module, a feature fusion module, a model building module, and an intelligent recognition module, wherein... The acquisition module is used to acquire multimodal data of patients with the six pathogenic factors in traditional Chinese medicine. The multimodal data includes facial video data, symptom data of the six pathogenic factors, and sympathetic nerve activity data. Through feature extraction, facial optical physiological change features are acquired based on the facial video data, symptom features are extracted based on the symptom data of the six pathogenic factors, and sympathetic nerve activity features are extracted based on the facial video data and the sympathetic nerve activity data. The feature fusion module is used to fuse the preprocessed facial optical physiological change features, the symptom features, and the sympathetic nerve activity features. The model building module is used to input the fused features into the interval judgment model built based on the KNN algorithm, and output the intelligent identification results of the six pathogenic factors in traditional Chinese medicine based on the interval judgment model. The intelligent identification module is used to output the corresponding syndrome differentiation and treatment plan for patients with the six pathogenic factors in traditional Chinese medicine based on the intelligent identification results of the six pathogenic factors in traditional Chinese medicine.

9. An electronic device, characterized in that, The device includes a processor, a communication bus, a user interface, a network interface, and a memory. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1 to 7.

Images