Multi-modal sensor-based traditional Chinese and western medicine fusion data analysis intelligent diagnosis system and method

Abstract

The invention discloses a traditional Chinese and western medicine fusion data analysis system and method based on a multi-mode sensor, and belongs to the technical field of intelligent medical treatment. The system comprises: (1) a tongue diagnosis module, which collects tongue images through a multispectral camera (400-1000nm), and extracts HSV color features (resolution of 0.1 degree) and LBP texture features (radius of 3 pixels); the pulse diagnosis module (2) adopts a laser Doppler sensor (850nm) to obtain a radial artery signal, and extracts pulse position / pulse potential / pulse shape three-dimensional features (sample entropy lt, 1.2 represents chordwise astringent pulse) through wavelet transform; and (3) the data processing module inputs the tongue vein features, fasting blood glucose (blood glucose + / -0.3 mmol / L precision), C-reactive protein and other western medicine biochemical indexes into a fusion decision model, dynamically distributes traditional Chinese medicine and western medicine weights (for example, the traditional Chinese medicine weight is smaller than or equal to 40% when the C-reactive protein is larger than 10 mg / L) through a typhoid miscellaneous disease rule base and a random forest algorithm, and generates physical feature vectors for health state classification. Organic cooperation of traditional Chinese medicine physical sign quantitative analysis and western medicine evidence-based medicine is achieved, the early-stage diabetes screening accuracy rate is improved by 23.7% (compared with single-mode diagnosis), and the method is suitable for family health monitoring and traditional Chinese and western medicine combined diagnosis and treatment.

Description

Intelligent Diagnostic System and Method for Integrative Traditional Chinese and Western Medicine Data Analysis Based on Multimodal Sensors Technical Field

[0001] This invention belongs to the field of intelligent medical technology, specifically involving a multimodal fusion data analysis system that integrates the collection of signs and symptoms from traditional Chinese medicine tongue diagnosis and pulse diagnosis with the analysis of humanistic indicators from Western medicine. It is applicable to family health monitoring, primary medical institutions, and integrated traditional Chinese and Western medicine diagnosis and treatment scenarios. Background Technology

[0002] Existing technological limitations: Single-modal diagnostic devices (such as tongue diagnosis instruments and pulse diagnosis instruments) cannot achieve complementary data between traditional Chinese medicine and Western medicine; traditional Chinese medicine relies on subjective experience and lacks quantitative standards; Western medicine testing equipment is expensive and cannot provide an overall assessment of physical condition. Summary of the Invention

[0003] Multimodal data fusion architecture sensor combination: Tongue diagnosis module: multispectral camera (400-1000nm wavelength) + ring polarized light source (to eliminate reflection interference) Pulse diagnosis module: laser Doppler flowmeter (VL53L0X) + piezoelectric thin film pressure sensor (MPXV7007DP) Western medicine indicator module: micro biochemical detection chip (blood glucose / blood lipids / CRP three-in-one) Data collaboration logic: The data collaboration logic of the system is shown in Figure 4 of the attached diagram in the manual.

[0004] 2. Dynamic Weight Allocation Model Rule Engine: def weight_adjustment(crp, glucose, tongue_score): if crp>10: # High inflammatory markers, Western medicine weight ↑ return {'western': 0.7, 'tcm': 0.3} elif tongue_score>0.8: # Significant tongue features, Traditional Chinese Medicine weight ↑ return {'western': 0.3, 'tcm': 0.7} else: # Default balanced weight return {'western': 0.5, 'tcm': 0.5} 3. Technical Solution Overview: The technical solution overview is shown in Figure 6 of the attached diagram in the manual.

[0005] Technical effects: (1) The diagnostic accuracy rate is increased to 91.2% (≥23% higher than that of a single TCM / Western medicine method); (2) The equipment cost of primary medical institutions is reduced by 65% ​​(compared to the traditional CT + TCM diagnostic combination). Detailed Implementation

[0006] 4. Specific Implementation Example 1: Tongue Image Acquisition and Processing The tongue was captured using an OV5640 camera (RGB+IR channel). ROI segmentation was performed using the OpenCV library to extract the HSV color histogram (H channel resolution 0.1°) and LBP texture features (radius 3 pixels, neighborhood 8 points).

[0007] Example 2: Pulse Signal Quantization. The VL53L0X sensor acquires radial artery displacement signals, which are decomposed into 5 levels of detail coefficients using wavelet transform (db4 basis function). The sample entropy of the 3rd level coefficients is calculated as the pulse complexity index. The piezoelectric thin film sensor must meet the following characteristics: Pressure sensitivity: 0.5–2.0 mV / kPa (preferably 1.0 mV / kPa); Linearity error: ≤±1.5%; FS resonant frequency: ≥1 kHz; Temperature drift: <0.05% / ℃ (according to standard: IEC 61000-4-21 electromagnetic compatibility test). Example 3: Dynamic Weight Allocation Logic. When a blood routine test shows a white blood cell count >10×10... 9 When the serum albumin level is <35g / L, the weight of Traditional Chinese Medicine (TCM) is reduced to 40%, and the Western medicine inflammation warning process is triggered first; when the tongue appearance characteristics match the constitution type B and the serum albumin level is <35g / L, the weight of the TCM treatment plan is increased to 80%. Example 4: Tongue characteristics for prediabetes screening: Yellow and greasy tongue coating (HSV hue value H=45±3°) Pulse characteristics: Slippery and rapid pulse (wavelet energy entropy > 2.5), CRP = 12mg / L Western medicine indicators: Fasting blood glucose 6.3mmol / L, HOMA-IR = 2.8 Dynamic weight: Western medicine weight 60% (abnormal blood glucose), Traditional Chinese medicine weight 40% Diagnostic output: Spleen deficiency and damp heat (Traditional Chinese medicine) + Insulin resistance warning (Western medicine) → Recommendation: Shenling Baizhu Powder + low dose of metformin (muscle-certified) Effect: Theoretically expected to improve accuracy by ≥20%, expected to reduce misdiagnosis rate in primary healthcare by 15%-25% Example 5: Tongue characteristics for chronic gastritis: Dark purple tongue (LBP variance > 120) Pulse characteristics: Wiry and hesitant pulse (sample entropy < 1.2) Western medicine indicators: Pepsinogen I / II = 5.2 Dynamic weight: Traditional Chinese medicine weight 70% (significant tongue and pulse characteristics), Western medicine weight 30% Diagnostic output: Liver and stomach disharmony (Traditional Chinese medicine) → It is recommended to use Chaihu Shugan San (Bupleurum Liver-Soothing Powder) combined with a gastric mucosa repair treatment. Figure descriptions: Figure 1 is a system structure block diagram; Figure 2 is a tongue image processing flowchart; Figure 3 is a pulse signal processing timing diagram; Figure 4 is a data collaboration logic diagram; Figure 5 is a dynamic weight allocation model rule engine diagram; Figure 6 is a technical solution overview diagram.

Claims

1. A smart diagnostic system and method for integrated traditional Chinese and Western medicine data analysis based on multimodal sensors, characterized in that, The process includes the following steps: Tongue diagnosis module: Collects multispectral data of the tongue image using a multispectral sensor, extracts HSV color features of the tongue coating (hue resolution ±0.1°) and LBP texture features of the tongue body; configured to acquire tongue images through a multispectral imaging unit (wavelength range 400-1000nm), and extracts HSV color space histogram features (H channel resolution ≤0.5°) and LBP texture features (neighborhood radius 3 pixels, 8 sampling points). Pulse diagnosis module: The system acquires radial artery laser Doppler signals using a laser Doppler sensor at a sampling rate of ≥60Hz, and extracts three-dimensional features of pulse position, pulse potential, and pulse shape through wavelet transform. It includes a laser Doppler blood flow detection unit (operating wavelength 850±10nm, sampling rate ≥60Hz) for acquiring radial artery displacement signals; a piezoelectric film pressure sensing unit (sensitivity ≥0.5mV / kPa, frequency response range 0.1-100Hz) for synchronously acquiring arterial pressure waveforms; and a data fusion processing module (dynamic weight classifier) ​​configured to input tongue image features, pulse wavelet energy entropy / sample entropy features, and Western medicine test (biochemical) indicators into a TCM-Western medicine integrated decision-making model (dynamic weight classifier) ​​to generate an 8-dimensional constitution feature vector for health status classification.

2. The mathematical expression for the physical feature vector $\vec{V}$ is defined as: $$ \vec{V} = [v1, v2, ..., v8] T = W \cdot \Phi(F) $$where: $W$ is the dynamic weight matrix (dimension 8×6), satisfying $\sum_{j=1}^{6} w_{ij} = 1$$\Phi(F)$ is the feature mapping function: $$ \Phi(F) =【】\begin{bmatrix} \text{HSV-H} \\ \text{LBP-Var} \\ \text{Wavelet-Entropy}\\ \text{Sample-Entropy} \\ \text{Glucose} \\ \text{CRP} \end{bmatrix} \xrightarrow{\text{standardization}} \begin{bmatrix} \phi_1 \\ \phi_2 \\ \phi_3 \\ \phi_4 \\ \phi_5 \\ \phi_6 \end{bmatrix} $$**Construction steps**:

1. Feature normalization: $\phi_i = \frac{f_i - 2. Dynamic weight allocation: $$ w_{ij} = \begin{cases}0.15 \times I_{\text{TCM}} + 0.85 \times I_{\text{WM}} & \text{if CRP>10} \\\text{RF\_Weight}(f_j) & \text{otherwise}\end{cases} $$ 3. Vector composition: $v_k = \sum_{j=1}^{6} w_{kj} \cdot \phi_j \quad (k=1,...,8)$ The TCM-Western medicine integrated decision model described above is constructed based on the symptom-syndrome rule base of "Treatise on Febrile Diseases" and the random forest algorithm, dynamically allocating TCM and Western medicine evidence-based weights. The dynamic weight classifier automatically assigns decision weights for Western and Traditional Chinese Medicine (Western medicine weight 30%-70%, Traditional Chinese Medicine weight 30%-70%) based on C-reactive protein levels and the significance of tongue features.

3. The method according to claim 1, characterized in that, The tongue diagnosis module includes a multispectral camera (such as OV5640) with a wavelength of 400-1000nm and a ring LED fill light (color temperature 5000K±200K, illuminance adjustable range 200-1000lux).

4. The method according to claim 1, characterized in that, The pulse diagnosis module's laser Doppler sensor (such as VL53L0X) acquires radial artery displacement signals at a sampling rate of 60Hz and uses a Butterworth low-pass filter (cutoff frequency 10Hz) for noise reduction.

5. The method according to claim 1, characterized in that, The dynamic weight allocation satisfies the following conditions: when C-reactive protein > 10 mg / L, the Western medicine decision weight automatically increases to ≥ 60%, and the Traditional Chinese Medicine weight decreases to below 40%; when the tongue appearance characteristics match constitution type A (tongue surface moisture < 40% and HSV-H value > 30), the weight allocation is adjusted accordingly. 0 Furthermore, when fasting blood glucose is >7.0 mmol / L, the weight of Western medicine evidence-based medicine increases to 70%.

6. When the tongue image HSV hue value H∈[40°,50°] and the LBP variance>100, the TCM decision weight is ≥60%.

7. The method according to claim 1, characterized in that, The Western medical chemical indicators include at least three of the following: fasting blood glucose, total cholesterol, and high-sensitivity C-reactive protein, with a detection accuracy requirement of blood glucose ±0.3 mmol / L and cholesterol ±5%.

8. The method according to claim 1, characterized in that, The laser Doppler sensor operates at a wavelength of 850±10nm and has a power density of <1mW / cm², conforming to the IEC 60825-1 human eye safety standard.

Images