Traditional Chinese medicine diagnosis and treatment decision-making method, system and device based on multi-modal data and medium

Abstract

The application provides a traditional Chinese medicine diagnosis and treatment decision method and system based on multi-modal data, equipment and a medium. The method is suitable for a traditional Chinese medicine intelligent auxiliary diagnosis and treatment system comprising a pulse diagnosis pen, and comprises the following steps: collecting pulse diagnosis pressure data of a user by the pulse diagnosis pen and extracting time domain features and spatial features to generate a pulse diagnosis vector; collecting tongue image, face image, audio signal and diagnosis answer to generate a multi-dimensional feature vector; mapping the multi-modal evidence source to a traditional Chinese medicine syndrome set, and generating an initial probability distribution result of each evidence source for each syndrome in the traditional Chinese medicine syndrome set; weighting and fusing the initial probability distribution result to generate a confidence distribution result of each evidence source for each syndrome in the traditional Chinese medicine syndrome set; and inputting the multi-modal evidence source and the confidence distribution result into a pre-constructed large language model to enable the large language model to generate a clinical judgment result and individualized diagnosis and treatment suggestion for the user. The application can improve the accuracy and reliability of online diagnosis and treatment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and healthcare technology, and in particular to methods, systems, devices and media for TCM diagnosis and treatment decision-making based on multimodal data. Background Technology

[0002] Online TCM consultations, as an important extension of traditional Chinese medicine services, effectively break down geographical limitations with their convenience and accessibility, providing efficient health assessments and conditioning guidance to the public. They play a crucial role in home self-examination, chronic disease management, and the popularization of TCM culture. Their core value lies in combining the four diagnostic methods of TCM (inspection, auscultation, inquiry, and palpation) with modern technology to meet the growing demand for personalized health management. Currently, common implementation methods for online TCM consultations fall into two main categories: one is mobile TCM analysis apps, which use cameras to capture tongue and facial images and microphones to record sound, performing single-dimensional analysis using conventional computer vision and speech recognition algorithms; the other is health management software that integrates multimodal data analysis, combining data such as heart rate, blood pressure, and vital signs questionnaires, and completing health assessments through preset rule engines or simple scoring card models. Both methods attempt to simplify TCM consultation processes and expand service coverage through technological means.

[0003] However, existing implementation methods have many prominent shortcomings, severely restricting the accuracy and clinical value of online TCM diagnosis and treatment. Mobile TCM analysis apps not only suffer from color distortion in image acquisition due to ambient lighting and device model limitations, and data quality issues stemming from a lack of standardized procedures and noise reduction control in audio acquisition, but also fail to achieve cross-modal correlation analysis due to their shallow and isolated machine learning models. Furthermore, they completely lack objective and quantifiable pulse diagnosis data, violating the holistic view of TCM's "four diagnostic methods combined." Health management software integrating multimodal data suffers from simple data fusion mechanisms, reliance on fixed rule bases, and a lack of quantitative management of multimodal data conflicts, resulting in insufficient personalization of analysis results, difficulty in uncovering complex syndrome relationships, and a "black box" risk in diagnostic logic when facing complex syndromes, leading to low clinical credibility. Similarly, they cannot comprehensively and objectively reproduce the essence of traditional TCM diagnostic methods such as the "three parts and nine pulses." Improving these shortcomings is not only crucial for enhancing the credibility of online TCM diagnosis and treatment and meeting clinical application needs, but also an inevitable requirement for promoting the standardization and universal accessibility of TCM diagnosis and treatment. Summary of the Invention

[0004] The present invention aims to provide a method, system, device and medium for TCM diagnosis and treatment decision making based on multimodal data, so as to solve the above-mentioned technical problems and improve the accuracy and reliability of online diagnosis and treatment.

[0005] To address the aforementioned technical problems, this invention provides a TCM diagnostic and treatment decision-making method based on multimodal data, applicable to TCM intelligent auxiliary diagnostic and treatment systems including pulse diagnosis pens, comprising: The pulse pressure data of the user is collected by the pulse pen and the temporal and spatial features of the pulse pressure data are extracted to generate a pulse vector. The pulse pen adopts a dynamic gain adjustment strategy and arranges several sensors in an asymmetric array structure to collect pulse pressure data. Collect users' tongue images, facial images, audio signals, and consultation answers, extract core features, and generate multidimensional feature vectors; among them, the multidimensional feature vectors include tongue diagnosis vectors, facial diagnosis vectors, audio diagnosis vectors, and consultation vectors; Using pulse diagnosis vectors and multidimensional feature vectors as multimodal evidence sources, the multimodal evidence sources are mapped to a pre-constructed set of TCM syndromes, and the initial probability assignment results of each evidence source to each syndrome in the TCM syndrome set are generated one by one; wherein, the TCM syndrome set contains several syndromes. The initial probability allocation results are weighted and fused according to the consistency measurement principle and the uncertainty measurement principle to generate the confidence allocation results of each evidence source for each syndrome in the TCM syndrome set. The multimodal evidence sources and confidence allocation results are input into the pre-built large language model so that the large language model can generate clinical judgment results and personalized diagnosis and treatment suggestions for users. The large language model is based on the LLM neural network model and is trained using a professional dataset in the TCM field that includes clinical guidelines and structured four diagnostic cases.

[0006] The above scheme, based on the asymmetric array sensor and dynamic gain adjustment strategy of the pulse diagnosis pen, accurately collects the spatiotemporal pressure characteristics of the radial artery, generating pulse diagnosis vectors containing temporal and spatial dimensions, overcoming the pain points of subjective and incomplete data in traditional pulse diagnosis. Simultaneously, it standardizes the collection of tongue, facial, audio, and consultation data, extracting their core features to generate multidimensional feature vectors, comprehensively covering the four diagnostic methods of Traditional Chinese Medicine (TCM), aligning with the holistic diagnostic principle of integrating the four methods, and providing high-quality data support for subsequent diagnosis and treatment. Multimodal evidence sources are mapped to a unified TCM syndrome set to generate initial probability allocation results. Then, weighted fusion is performed using consistency and uncertainty measurement principles, effectively handling conflicts and uncertainties in heterogeneous data, improving the objectivity and reliability of syndrome judgment, and overcoming the diagnostic bias problems caused by traditional single-modal analysis or simple weighted fusion. Finally, the confidence allocation results and multimodal evidence sources are input into a large model trained on a professional TCM dataset. Its dual processing engine can adapt to different complex cases, generating personalized treatment suggestions. By deeply integrating traditional Chinese medicine diagnostic and treatment thinking with artificial intelligence and sensor technology, users can obtain professional-level TCM health assessments through mobile terminals, improving the accuracy and reliability of online diagnosis and treatment.

[0007] In one implementation, the pulse pen employs a dynamic gain adjustment strategy and arranges several sensors in an asymmetric array structure to collect pulse pressure data, specifically including: The pulse diagnosis pen includes several pressure sensors; the sensors are arranged in an array structure, and the central area of ​​the array is left empty. Static pressure calibration is performed based on the central area of ​​the array to align with the radial artery pulsation center point. The pressure sensors, arranged in an array structure, cover the lateral and longitudinal fluctuation range of the radial artery to collect the user's pulse pressure data. Based on the threshold range of the average static pressure monitoring value of the pressure sensor in the array structure, the pulse diagnosis pen switches between pulse-taking states and gain modes; the pulse-taking states include floating pulse, deep pulse and middle pulse, and the gain modes include high gain mode, low gain mode and default gain mode.

[0008] In one implementation, pulse pressure data from the user is collected using a pulse pen, and the temporal and spatial features of the pulse pressure data are extracted to generate a pulse vector, specifically including: The pulse pressure data of the user is continuously collected based on a preset sampling rate; the pulse pressure data includes the ADC value and collection timestamp of each sensor. The ADC values ​​are mapped to the actual pressure values, and the pressure values ​​are reconstructed into a pressure heatmap based on the interpolation algorithm; the pressure heatmap is used to characterize the diameter, length, and pulsation intensity of the radial artery. The pressure heatmaps are sorted according to the acquisition timestamps, and then converted into a four-dimensional spacetime tensor to obtain the pulse diagnosis vector. The expression for the four-dimensional spacetime tensor is as follows: ; In the formula, This represents the number of pulse pressure data samples collected in a single session. The time step represents the number of consecutively acquired time frames. These are characteristic channels, including AC channels and DC channels; This represents the vertical pixel dimension of the pressure heatmap, corresponding to the vertical distribution of the radial artery. This represents the horizontal pixel dimension of the pressure heatmap, corresponding to the horizontal distribution of the radial artery.

[0009] In one implementation, the user's tongue image, facial image, audio signal, and medical consultation answers are collected, and core features are extracted to generate a multi-dimensional feature vector, specifically including: The tongue image is color-corrected and the color-corrected tongue region is extracted. The color and texture features of the tongue region are extracted using the LAB color model to obtain the tongue diagnosis vector. Identify the facial reflection areas in the facial image, and extract the gloss and color deviation of the facial reflection areas using the LAB color model to obtain the facial diagnosis vector; MFCC features are extracted from the audio signal, and a voiceprint feature matrix is ​​constructed based on the MFCC features. The denoised voiceprint feature matrix is ​​used as the voice diagnosis vector. The audio signal is the sound data of the user reading a preset five-tone character sample. The consultation answers are mapped to Boolean vectors or degree vectors to obtain the consultation vector; where the consultation answers are the user's responses to several body mass scales that follow the standards of the China Association of Traditional Chinese Medicine.

[0010] In one implementation, before weighting and fusing the initial probability allocation results according to the consistency metric principle and the uncertainty metric principle, the initial probability allocation results are updated based on the initial confidence scores of the multimodal evidence sources. Specifically: The sharpness scores of the tongue and face images are calculated using preset operators, and the sharpness scores are used as the initial confidence scores of the images. The initial confidence score of the pulse pressure data was calculated based on the baseline drift and signal-to-noise ratio of the pulse pressure data. The initial probability assignment results corresponding to modal evidence sources with initial confidence scores below a preset threshold are subjected to maximum entropy fuzzification to obtain updated initial probability assignment results; wherein, the initial confidence score is used to characterize the data collection quality.

[0011] In one implementation, the initial probability allocation results are weighted and fused according to the consistency measurement principle and the uncertainty measurement principle to generate the confidence allocation results of each evidence source for each syndrome in the TCM syndrome set, specifically including: Based on the consistency measurement principle, the cosine similarity between each pair of evidence sources is calculated one by one; whereby the cosine similarity is used to characterize the consistency between the two evidence sources in the direction of symptom judgment, and the expression for the cosine similarity is: ; In the formula, For the first The source of evidence and the first Cosine similarity between evidence sources; For the first The initial probability assignment results of each source of evidence; For the first The initial probability assignment results of each source of evidence; The total number of syndromes in the TCM syndrome differentiation; The first in the collection of syndromes in traditional Chinese medicine Individual symptoms; for The source of evidence for the first The initial probability allocation results for each syndrome; For the first The source of evidence for the first The initial probability allocation results for each syndrome; The aggregate support is generated for each source of evidence based on cosine similarity; where aggregate support characterizes the total degree of support from other modal evidence sources for the current source of evidence, and the expression for aggregate support is: ; In the formula; For the first Aggregate support of each source of evidence; The modality number of the evidence source; For traversal index; Calculate the Dunk entropy for each source of evidence. Generate corrected weights based on the aggregated support, Dunk entropy, and initial confidence score of each source of evidence. Then, based on these corrected weights, generate the confidence assignment results for each source of evidence to each syndrome in the TCM syndrome set. The expressions for Dunk entropy and corrected weights are as follows: ; In the formula, For the first Deng's entropy of one source of evidence; Collection of Traditional Chinese Medicine Syndromes A subset, each subset representing a combination of syndromes, which can be a single syndrome or a combination of multiple syndromes; The number of symptoms contained in the subset; For the first The initial probability assignment results of each source of evidence to subset A; ; In the formula, For the first Adjusted weights for each source of evidence; This is the normalization function; , , These are custom weighting coefficients; For the first Initial confidence scores for each source of evidence; For the first The pre-defined weights for each source of evidence in traditional Chinese medicine (TCM) clinical practice are set based on TCM clinical experience.

[0012] In one implementation, the confidence level allocation results for each syndrome in the TCM syndrome set are generated based on modified weights, specifically including: The initial probability allocation result for each evidence source to each subset is obtained one by one. The initial probability allocation results for subsets with conflicting symptom elements are then corrected to obtain the corrected probability allocation result for each evidence source. The expression for the correction function is as follows: ; In the formula, This is the corrected initial probability assignment result for the i-th evidence source to subset A; For the first Adjusted weights for each source of evidence; For the first The initial probability assignment results of each source of evidence to subset A; Collection of Traditional Chinese Medicine Syndromes A subset, each subset representing a combination of syndromes, which can be a single syndrome or a combination of multiple syndromes; For the revised i-th source of evidence to the TCM syndrome set Trust level; The modified probability allocation results are subjected to orthogonal sum operations a preset number of times using Dempster's synthesis rules to generate confidence allocation results for each syndrome in the TCM syndrome set from each evidence source; where the preset number of operations is k-1, and k is the modality number of the evidence source.

[0013] Secondly, this application also provides a TCM intelligent auxiliary diagnosis and treatment system, including: A pulse diagnosis pen and a host computer; wherein the output end of the pulse diagnosis pen is connected to the input end of the host computer, and the host computer deploys intelligent programs and a large language model; The pulse diagnosis pen is used to collect the user's pulse pressure data and transmit it to the host computer. The host computer is used to execute the TCM diagnosis and treatment decision-making method based on multimodal data as described above; wherein, the intelligent program is used to collect the user's tongue image, facial image, audio signal and diagnosis answer.

[0014] The above solution achieves a deep integration of traditional Chinese medicine diagnosis and treatment with intelligent technology through a collaborative architecture of a pulse diagnosis pen and a host computer. The pulse diagnosis pen, with its professional hardware design, accurately collects pulse pressure data, overcoming the shortcomings of traditional pulse diagnosis, which is subjective and difficult to quantify, thus providing objective evidence for diagnosis. The host computer, through intelligent programs, completes multimodal data including tongue appearance, facial appearance, audio, and medical history, fully covering the four diagnostic dimensions of traditional Chinese medicine and aligning with the core principle of integrated diagnosis. Simultaneously, the host computer executes a dedicated diagnostic algorithm, combining a deeply fine-tuned large language model to complete data fusion and diagnostic reasoning. This effectively handles conflicts from multiple data sources, reduces uncertainty, and generates personalized treatment suggestions. The efficient collaboration of these components preserves the essence of traditional Chinese medicine diagnosis while improving accuracy and efficiency through standardized processes, balancing medical professionalism with ease of use.

[0015] Thirdly, this application also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the TCM diagnosis and treatment decision-making method based on multimodal data as described above.

[0016] Fourthly, this application also provides a computer-readable storage medium, which includes a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the TCM diagnosis and treatment decision-making method based on multimodal data as described above. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a TCM diagnosis and treatment decision-making method based on multimodal data provided in one embodiment of the present invention; Figure 2 This is a schematic diagram of a module of a traditional Chinese medicine intelligent auxiliary diagnosis and treatment system provided in one embodiment of the present invention; Figure 3 This is a physical diagram of a traditional Chinese medicine intelligent auxiliary diagnosis and treatment system provided in one embodiment of the present invention; Figure 4 This is a schematic diagram of the homepage of a smart program provided in one embodiment of the present invention. Detailed Implementation

[0018] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] The terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] First, some of the terms used in this application will be explained to facilitate understanding by those skilled in the art.

[0022] (1) LLM Neural Network Model: Large Language Model is a large-scale pre-trained language model based on deep learning. With Transformer as the core architecture, it learns the syntax, semantics, logic, knowledge and context of human language through self-supervised pre-training on massive text corpora, and has powerful text understanding, generation, reasoning, dialogue and knowledge retrieval capabilities.

[0023] (2) ADC value: Analog-to-Digital Converter refers to the digital quantization value output by the analog-to-digital converter, which is the value after converting a continuous analog signal (such as voltage or pressure) into a discrete digital signal.

[0024] (3) MFCC: Mel-Frequency Cepstral Coefficients, a speech feature extraction method that simulates the auditory characteristics of the human ear, converting sound signals into frequency features that conform to auditory perception.

[0025] Example 1 See Figure 1 ,picture Figure 1 This is a flowchart illustrating a TCM diagnosis and treatment decision-making method based on multimodal data according to an embodiment of the present invention. The embodiment of the present invention provides a TCM diagnosis and treatment decision-making method based on multimodal data, applicable to a TCM intelligent auxiliary diagnosis and treatment system including a pulse diagnosis pen, comprising steps 101 to 104, each step being as follows: Step 101: Collect pulse pressure data from the user using a pulse pen and extract the temporal and spatial features of the pulse pressure data to generate a pulse vector; wherein, the pulse pen uses a dynamic gain adjustment strategy and arranges several sensors in an asymmetric array structure to collect pulse pressure data.

[0026] In this embodiment of the invention, a pulse diagnosis pen equipped with an asymmetric array sensor collects the user's pulse pressure data. A dynamic gain adjustment strategy is employed to ensure data quality under different pressure levels. The temporal and spatial features of the data are extracted, and a pulse diagnosis vector is ultimately generated. The asymmetric array sensor and the dynamic gain adjustment strategy can solve the problems of low spatial resolution and insufficient simulation of pressure intensity in traditional pulse diagnosis devices, enabling objective and quantitative collection of pulse diagnosis information from the three pulse positions and nine pulse points.

[0027] In one embodiment, the pulse diagnosis pen includes several pressure sensors. The sensors are arranged in an array structure, with a sensor gap in the central area of ​​the array. Static pressure calibration is performed based on the radial artery pulsation center point in the central area of ​​the array. The pressure sensors arranged in the array structure cover the lateral and longitudinal fluctuation range of the radial artery to collect the user's pulse pressure data. The pulse diagnosis pen switches between pulse-taking states and gain modes according to the threshold range of the average static pressure monitoring value of the pressure sensors in the array structure. The pulse-taking states include floating pulse, deep pulse, and medium pulse, and the gain modes include high gain mode, low gain mode, and default gain mode.

[0028] In this embodiment of the invention, the pulse pen employs a 5×5-1 asymmetric array of pressure sensors, a design specifically adapted to the spatial acquisition requirements of traditional Chinese medicine pulse diagnosis. 5×5 represents a theoretical 5x5 sensor matrix, and -1 indicates the removal of one sensor from the center, leaving 24 high-precision capacitive pressure sensors forming the actual acquisition array. The central empty area is not a design flaw, but rather used to locate the core pulsation point of the radial artery and perform static pressure calibration, ensuring a unified acquisition benchmark for the entire array. This asymmetric arrangement guarantees sensor density and, through spatial layout optimization, allows the array to fully cover the radial artery in both the width direction and the distribution directions of the cun, guan, and chi segments—that is, the lateral and longitudinal pulsation range—avoiding omissions in key areas due to uniform sensor distribution. Based on the above array structure, the pulse pen synchronously acquires the user's radial artery pressure data through the sensors. During acquisition, the user needs to place the pulse pen against the radial artery pulsation area on the inside of the wrist; the sensor array will simultaneously capture the pressure changes at different time points at the three key points of cun, guan, and chi. The 5×5-1 array design, through precise spatial resolution specifications, can improve the array spatial resolution to 1.378 mm, while maintaining a sensor sensitivity ≥2.42 kPa. - ¹ With an accuracy of ≤0.5%FS, it can accurately capture subtle differences in radial artery pulsation, including variations in vessel diameter, pulsation intensity, and rhythm regularity. During data acquisition, the sensor records pressure data in real time at a sampling rate of 100Hz and transmits it to the software via BLE Low Energy Bluetooth to ensure data real-time performance and integrity. The pulse pen determines the current pulse-taking state based on the average static pressure monitoring value of all sensors in the array and automatically switches the corresponding gain mode. The core purpose is to obtain a high signal-to-noise ratio pulse signal through an adaptive filtering network under different pressure levels. When the pressure monitoring value... When the pressure reading is approximately 37.5 mmHg (corresponding to a light pressure state), it is determined to be a floating pulse state, and the system switches to high-gain mode, with the gain linearly increasing to the 200-500x range. This mode amplifies weak surface pulsation signals through a non-linear gain compensation mechanism, while differential preamplification and bandpass filtering cancel out environmental noise and human body common-mode interference, avoiding feature loss due to signal attenuation. When the pressure monitoring value is within the range... When the pressure is detected, it is determined to be in a pulse state, and the default gain mode is activated. To avoid frequent gain fluctuations caused by slight hand tremors, a "hysteresis comparison" strategy can be used, fine-tuning the gain only when the pressure fluctuation exceeds ±10%, ensuring the stability and consistency of signal acquisition. When the pressure monitoring value... When the pressure is approximately 150 mmHg (corresponding to a heavy press), it is determined to be a deep pulse state, and the system switches to low gain mode, reducing the gain to the range of 10-50 times. Through bidirectional diode clamping protection and automatic gain attenuation feedback, the input voltage range is limited to prevent signal saturation distortion caused by heavy press, ensuring complete capture of deep pulsation signals.

[0029] For example, the user places the pulse pen against the radial artery area on the inside of the wrist. The 24 sensors in the 5×5-1 asymmetric array naturally cover the longitudinal areas of the cun, guan, and chi points and the transverse width of the radial artery. The central blank area is aligned with the point of strongest pulsation at the guan point for positioning calibration. The pulse pen begins to collect pressure data at a sampling rate of 100Hz. In the initial stage, the pressure applied is relatively light, and the average static pressure of the array is... =3.2kPa, the system determines it to be a floating pulse state and automatically switches to high gain mode, with the gain multiplier set according to G=K / The calculation was adjusted to 480 times, and a bandpass filter was activated to filter out environmental electromagnetic noise and accurately capture weak surface pulsation signals. As the pressure increased... When the pressure increased to 11.5 kPa, the system determined it to be in a pulse state and switched to the default gain of 100x. During this period, slight hand tremors caused pressure fluctuations of ±7%, but this did not trigger gain adjustment, maintaining stable signal acquisition. If the pressure was further increased... Upon reaching 22.8 kPa, the system identified it as a deep pulsation state and switched to a 35x low-gain mode. A bidirectional clamping circuit limited the input voltage to ±0.7V. The MCU monitored the op-amp output voltage in real time, which approached 3.0V, triggering an interrupt within 0.8ms. The gain decreased by 3dB to 24.7 times to prevent signal clipping distortion and fully capture the deep pulsation characteristics. The entire acquisition process lasted 30 seconds, with 24 sensors simultaneously recording pressure data.

[0030] In one embodiment, pulse pressure data of a user is collected using a pulse pen, and the temporal and spatial features of the pulse pressure data are extracted to generate a pulse vector. Specifically, this includes: continuously collecting the user's pulse pressure data based on a preset sampling rate; wherein the pulse pressure data includes the ADC value and collection timestamp of each sensor; mapping the ADC values ​​to actual pressure values ​​and reconstructing the pressure values ​​into a pressure heatmap based on an interpolation algorithm; wherein the pressure heatmap is used to characterize the thickness, length, and pulsation intensity of the radial artery; sorting the pressure heatmaps according to the collection timestamps, and converting the sorted pressure heatmaps into a four-dimensional spatiotemporal tensor to obtain the pulse vector; wherein the expression of the four-dimensional spatiotemporal tensor is: ; In the formula, This represents the number of pulse pressure data samples collected in a single session. The time step represents the number of consecutively acquired time frames. These are characteristic channels, including AC channels and DC channels; This represents the vertical pixel dimension of the pressure heatmap, corresponding to the vertical distribution of the radial artery. This represents the horizontal pixel dimension of the pressure heatmap, corresponding to the horizontal distribution of the radial artery.

[0031] In this embodiment of the invention, the pulse pen continuously collects pressure data from the user's radial artery at a preset sampling rate. The collected raw data contains two core types of information: first, the ADC values ​​output by each of the 24 sensors, directly reflecting the pressure intensity sensed by the sensors; and second, the collection timestamp, accurately recording the collection time of each set of data to ensure the accuracy of the time sequence. The core of this step is to ensure the continuity and integrity of the data. By capturing subtle dynamic changes in the pulse wave through high-frequency sampling, it provides a foundation for subsequent time-domain feature extraction. At the same time, the timestamp provides a crucial basis for the temporal sorting and tensor transformation of the data, avoiding the impact of data corruption on feature accuracy. Then, an analytical algorithm maps the original ADC values ​​to actual physical pressure values, achieving a quantization conversion from digital signals to actual pressure and eliminating numerical deviations caused by differences in sensor hardware. Subsequently, a bicubic interpolation algorithm is used to reconstruct the pressure values ​​of 24 discrete sensors, generating a 24×24 pixel pressure heatmap. The bicubic interpolation algorithm can accurately calculate the pressure distribution of the entire radial artery coverage area from the pressure values ​​of adjacent discrete points, allowing the heatmap to clearly present the thickness, length, and pulsation intensity of the radial artery. It should be noted that the pulsation intensity is represented by pixel color, and the depth of pixel color corresponds to the pressure magnitude. This step transforms discrete point data into continuous two-dimensional image data, intuitively restoring the spatial pressure distribution characteristics of the radial artery, providing a visual and computable carrier for spatial feature extraction. Furthermore, based on the acquisition timestamp, the continuously acquired pressure heatmaps are first sorted by time... The data are arranged sequentially to form a time-series heatmap sequence. This sequence is then converted into a four-dimensional spatiotemporal tensor, ultimately generating a pulse diagnosis vector. The tensor parameters are strictly adapted to the data characteristics: B represents the batch size (1 if data is collected from only one user at a time); T represents the time step, carrying temporal features such as pulse rate and rhythm (30 seconds for a 100Hz sampling rate, resulting in 3000 frames); C represents the feature channels, containing two channels: AC stores dynamic pulse wave changes, and DC stores static pressure values, corresponding to dynamic pulsation features and pressure depth features, respectively; H represents vertical pixels, and W represents horizontal pixels, both 24, consistent with the pressure heatmap size, corresponding to the longitudinal and transverse spatial distribution of the radial artery. This step integrates temporal, spatial, and feature channel information, transforming the pressure data into a standardized vector form that can be parsed by a deep learning model.

[0032] For example, the pulse pen collects data at a sampling rate of 100Hz, generating 3000 sets of raw data within 30 seconds. Each set contains the ADC values ​​of 24 sensors and corresponding timestamps, such as 0.01 seconds, 0.02 seconds, ... 30.00 seconds. The software maps the ADC values ​​to pressure values; for example, a sensor ADC value of 2048 corresponds to a pressure of 10 kPa. Then, using a bicubic interpolation algorithm, the 24 discrete pressure points are reconstructed into a 24×24 pixel pressure heatmap. In the heatmap, the pixels corresponding to the cun position are darker, followed by the guan position, and then the chi position is lighter. The corresponding pressure value ranges are 12-15 kPa, 10-13 kPa, and 8-11 kPa, clearly showing the spatial distribution of the radial artery with the highest pressure at the cun position and the lowest at the chi position. The 3000 frames of heatmaps are sorted by timestamp and converted into a four-dimensional spatiotemporal tensor. Where B=1, T=3000, C=2 (AC channel stores pulsation variation, DC channel stores average pressure), H=24, W=24, ultimately forming Tensor∈R 1×3000×2×24×24 A standardized pulse diagnosis vector. This vector contains both the rhythmic changes of the pulse over 30 seconds and preserves the spatial pressure distribution at the cun, guan, and chi positions, and can be directly input into subsequent models for analysis.

[0033] Step 102: Collect the user's tongue image, facial image, audio signal, and consultation answers, and extract core features to generate a multi-dimensional feature vector; among which, the multi-dimensional feature vector includes tongue diagnosis vector, facial diagnosis vector, audio diagnosis vector, and consultation vector.

[0034] In this embodiment of the invention, tongue images, facial images, audio signals, and consultation answers are collected from the user via a mobile terminal. Core features are extracted using U-Net networks, the MediaPipe framework, and the MFCC algorithm, respectively, generating corresponding tongue diagnosis vectors, facial diagnosis vectors, audio diagnosis vectors, and consultation vectors, forming a multi-dimensional feature vector. By combining a standardized acquisition process with professional feature extraction algorithms, the shortcomings of traditional apps—such as image color distortion, audio noise interference, and superficial analysis—are overcome, ensuring the reliability of the observation, auscultation, and inquiry diagnosis data.

[0035] In one embodiment, the user's tongue image, facial image, audio signal, and consultation answers are collected, and core features are extracted to generate a multi-dimensional feature vector. Specifically, this includes: color-correcting the tongue image and extracting the color-corrected tongue region; extracting the color and texture features of the tongue region using the LAB color model to obtain a tongue diagnosis vector; identifying the facial reflection area in the facial image; extracting the gloss and color deviation of the facial reflection area using the LAB color model to obtain a facial diagnosis vector; extracting the MFCC features of the audio signal; constructing a voiceprint feature matrix based on the MFCC features; and using the denoised voiceprint feature matrix as a voice diagnosis vector. The audio signal is the sound data of the user reading a preset five-tone character sample. The consultation answers are mapped to Boolean vectors or degree vectors to obtain consultation vectors. The consultation answers are the user's responses to several body weight scales conforming to the standards of the China Association of Traditional Chinese Medicine.

[0036] In this embodiment of the invention, the core of tongue diagnosis vector generation is to eliminate environmental interference and accurately extract the core features of the tongue. First, the system uses background color temperature and color rendering information obtained from built-in or external devices to perform color correction on the user's captured tongue image, solving the problem of tongue color distortion under different lighting conditions and ensuring the objectivity of the tongue body and coating color. Next, the U-Net convolutional neural network architecture is used to perform semantic segmentation on the corrected image, accurately extracting the tongue region and removing background interference such as oral mucosa and teeth, laying the foundation for subsequent feature analysis. Finally, based on the LAB color model, color and texture features of the tongue region are extracted. Color features include the mean values ​​of the L, A, and B channels, corresponding to traditional Chinese medicine descriptions of tongue colors such as pale white and reddish-brown; texture features include texture entropy values, used to reflect the degree of greasiness of the tongue coating. These features are integrated into a vector form, i.e., the tongue diagnosis vector. Facial diagnosis vector generation focuses on feature extraction from the facial organ reflex zones to align with the traditional Chinese medicine theory of "face corresponding to organs." Using Google's MediaPipe facial recognition framework, facial landmarks in the user's facial image are quickly detected and aligned. Based on this, regardless of the user's shooting posture or phone angle, the system can accurately locate the TCM facial organ reflex zones, such as the glabella and the tip of the nose. Subsequently, the LAB color model is used to analyze the color of these specific reflex zones, extracting two core features: first, glossiness based on HSV-V value conversion, which reflects the functional state of the organs; bright gloss generally indicates normal function, while dullness may indicate imbalance; second, color deviation, used to quantify the difference between the color of the reflex zone and the standard healthy color. These features are integrated into a vector to form the facial diagnosis vector. The voice diagnosis vector generation focuses on feature extraction that simulates human ear perception, adapting to the TCM auscultation and olfaction assessment requirements for timbre and sound quality. The user reads a preset five-tone character sample through the phone's microphone, corresponding to the five-tone theory of TCM (Gong, Shang, Jiao, Zhi, Yu), and the corresponding audio signal is collected. Next, the audio signal undergoes noise reduction processing to filter out environmental noise interference and retain the pure speech signal. Then, the MFCC (Melbourne Frequency Cepstral Coefficient) feature of the audio is extracted. This feature simulates the nonlinear perception of frequency and loudness by the human ear, focusing on the core information of timbre and sound quality related to TCM auscultation. It constructs a voiceprint feature matrix by extracting 12 key amplitude and frequency parameters. Finally, the noise-reduced voiceprint feature matrix is ​​used as the voice diagnosis vector, providing standardized acoustic features for subsequent analysis. The key to generating the diagnosis vector is converting the user's subjective feedback into an objective and calculable vector form. The system follows the standards of the China Association of Traditional Chinese Medicine, designing a body weight scale that includes dimensions such as physical characteristics, environmental adaptability, physiological performance, and psychological characteristics. Users provide their answers via mobile terminals. Subsequently, the answers are standardized and mapped: for "yes / no" questions, a 0-1 Boolean vector is mapped, with "yes" = 1 and "no" = 0; for "degree rating" questions, a 0-10 degree vector is mapped, such as excellent = 10, good = 8, average = 5, and poor = 2.All mapped vectors are integrated to form a complete consultation vector, ensuring that the consultation data meets the computational requirements of multimodal fusion.

[0037] For example, a user takes a picture of their tongue under natural light near a window. After color correction, the tongue region is segmented using U-Net, and LAB color features (L=72, A=4.5, B=10.2) and texture entropy (1.3) are extracted and integrated into a tongue diagnosis vector [72, 4.5, 10.2, 1.3]. For the captured facial image, the MediaPipe framework aligns the user's facial landmarks, locking in reflective areas such as the glabella and tip of the nose. The glossiness (0.85) and color deviation (0.12) of the lung area and the glossiness (0.78) and color deviation (0.15) of the spleen area are extracted, generating a facial diagnosis vector [0.85, 0.12, 0.78, 0.15]. Voice data of users reading aloud samples of the five tones (Gong, Shang, Jiao, Zhi, Yu) were collected. After audio noise reduction, 12 MFCC features were extracted to construct a voiceprint feature matrix. The simplified voice diagnosis vector is [0.32, 0.15, 0.28, 0.41, 0.35, 0.22, 0.18, 0.25, 0.30, 0.16, 0.23, 0.29]. The answers from the user's feedback in the body weight scale were mapped: whether the user felt cold (Yes = 1), sleep quality (Good = 8), whether the user's stool was sticky (No = 0), and whether the user was irritable (Mild = 3). The mapped data generated the consultation vector [1, 8, 0, 3].

[0038] Step 103: Using pulse diagnosis vector and multidimensional feature vector as multimodal evidence sources, map the multimodal evidence sources to a pre-constructed set of TCM syndromes, and generate the initial probability assignment results of each evidence source to each syndrome in the TCM syndrome set one by one; wherein, the TCM syndrome set contains several syndromes.

[0039] In this embodiment of the invention, pulse diagnosis vectors and multidimensional feature vectors are used as multimodal evidence sources, and a unified TCM syndrome set is pre-constructed. A Softmax classifier or fuzzy membership function is used to map each evidence source to the syndrome set, generating an initial probability assignment result for each syndrome for each evidence source. The unified TCM syndrome set provides a consistent mapping target for heterogeneous multimodal data, solving the problem of direct fusion of different types of data. The initial probability assignment transforms physical features into syndrome confidence levels, establishing a standardized computational foundation for subsequent weighted fusion and improving the objectivity of syndrome differentiation.

[0040] Specifically, multimodal evidence sources form the foundation for subsequent mapping and allocation. The core includes two key vectors: first, the pulse diagnosis vector, a four-dimensional spatiotemporal tensor generated from the temporal and spatial features extracted from pulse pressure data, carrying objective quantitative data from traditional Chinese medicine's "palpation"; second, the multidimensional feature vector, integrating the core features of inspection, auscultation, and inquiry, comprehensively covering the information from the four diagnostic methods of traditional Chinese medicine. These two types of vectors reflect the user's health status from different dimensions, jointly constituting a multimodal evidence system, ensuring the comprehensiveness of the evidence sources and aligning with the holistic diagnostic principle of integrating the four diagnostic methods of traditional Chinese medicine. The TCM syndrome set is a standardized set of syndromes pre-constructed based on TCM theory, clinical guidelines, and expert consensus, serving as a unified mapping target for multimodal evidence sources. The syndrome set must explicitly include core TCM syndromes, avoiding ambiguity or repetition. The TCM syndrome set covers common basic syndrome types in TCM clinical practice. The core value of constructing a unified syndrome set is to solve the semantic alignment problem of heterogeneous multimodal data, allowing different types of evidence, such as tongue diagnosis and pulse diagnosis, to be mapped to the same standardized syndrome system, providing a prerequisite for subsequent cross-modal fusion analysis. The core of the mapping process is to transform the feature vectors of each evidence source into initial confidence levels for each syndrome in the syndrome set, i.e., initial probability allocation results, using a specific algorithm. Specifically, for each evidence source—tongue diagnosis, facial diagnosis, pulse diagnosis, voice diagnosis, and questioning—the vectors are mapped using either a Softmax classifier or a fuzzy membership function. Taking core features from the vectors, such as the LAB color features of tongue diagnosis and the spatiotemporal tensor features of pulse diagnosis, as input, the algorithm combines knowledge from the field of Traditional Chinese Medicine (TCM), such as a red tongue often indicating Yin deficiency and a wiry pulse often indicating blood stasis, to calculate the probability that the evidence source supports the truth of each syndrome. The initial probability allocation result for each evidence source must satisfy the condition that "the sum of the probabilities of all syndromes is 1," while allowing a partial allocation of probabilities to the entire TCM syndrome set. In this case, the entire TCM syndrome set is treated as an unknown set to handle data uncertainty.

[0041] For example, suppose we have obtained the user's multimodal evidence sources: tongue diagnosis, facial diagnosis, pulse diagnosis, voice diagnosis, and questioning vectors, and the TCM syndrome set is as follows: The multimodal evidence sources include tongue diagnosis vectors [72, 4.5, 10.2, 1.3], facial diagnosis vectors [0.85, 0.12, 0.78, 0.15], pulse diagnosis vectors (a four-dimensional tensor of 1×3000×2×24×24), voice diagnosis vectors [0.32, 0.15, ..., 0.29], and questioning vectors [1, 8, 0, 3]. The tongue diagnosis vectors were analyzed using a Softmax classifier, yielding tongue body LAB color features L=72, A=4.5, B=10.2, which conforms to the characteristics of Yin deficiency tongue color, generating initial probability assignment results. , , , , , , Based on the spatiotemporal characteristics of the pulse diagnosis tensor (such as a slow pulse rate and uneven pressure distribution at the cun, guan, and chi positions, consistent with blood stasis syndrome), initial probability assignment results are generated through fuzzy membership function mapping: , The pulse diagnosis result was normal (m = 0.05). 10. Similarly, the initial probability allocation results of the face diagnosis vector and the questioning vector are generated, and finally the initial probability allocation results of the 5 evidence sources for the 6 syndromes are obtained. Each result satisfies the probability sum of 1, which provides a standardized trust basis for subsequent weighted fusion.

[0042] Step 104: The initial probability allocation results are weighted and fused according to the consistency measurement principle and the uncertainty measurement principle to generate the confidence allocation results of each evidence source for each syndrome in the TCM syndrome set. The multimodal evidence sources and confidence allocation results are input into the pre-built large language model so that the large language model can generate clinical judgment results and personalized diagnosis and treatment suggestions for users. The large language model is based on the LLM neural network model and is trained using a professional dataset in the TCM field that includes clinical guidelines and structured four diagnostic cases.

[0043] In this embodiment of the invention, the initial probability allocation results are weighted and fused based on consistency and uncertainty metrics to obtain the confidence allocation results for each syndrome. The multimodal evidence sources and confidence results are then input into a large language model trained on a professional dataset in the field of Traditional Chinese Medicine (TCM) to generate clinical judgment results and personalized treatment suggestions. The weighted fusion of consistency and uncertainty effectively handles evidence conflicts, highlights the role of highly credible evidence, and improves the accuracy and robustness of diagnostic conclusions. The large-scale model specifically designed for the TCM field possesses deep reasoning capabilities; combined with training on a professional dataset, it can uncover complex syndrome relationships, generate personalized and compliant treatment suggestions, and ensure medical safety.

[0044] In one embodiment, before weighting and fusing the initial probability allocation results according to the consistency measurement principle and the uncertainty measurement principle, the method further includes updating the initial probability allocation results based on the initial confidence scores of the multimodal evidence sources. Specifically: the clarity scores of the tongue image and facial image are calculated using a preset operator, and the clarity scores are used as the initial confidence scores of the images; the initial confidence scores of the pulse pressure data are calculated based on the baseline drift and signal-to-noise ratio of the pulse pressure data; the initial probability allocation results corresponding to the modal evidence sources with initial confidence scores lower than a preset threshold are subjected to maximum entropy fuzzification processing to obtain the updated initial probability allocation results; wherein, the initial confidence scores are used to characterize the data acquisition quality.

[0045] The initial confidence score for image-based evidence sources such as tongue and facial images is primarily a measure of data acquisition quality. In this embodiment, the Laplacian operator is used, which effectively detects the edge sharpness of an image, indirectly reflecting its clarity. The sharper the edges, the less interference from ambient light and camera shake, indicating higher data quality and a higher clarity score. Laplacian operator convolution operations are performed on tongue and facial images respectively. The clarity score is quantified by statistically analyzing the variance or grayscale distribution of the results. This score is the initial confidence score for the image-based evidence source, ranging from [0,1], with values ​​closer to 1 indicating better data quality. The initial confidence score for pulse pressure data is calculated using two core indicators: baseline drift and signal-to-noise ratio. Baseline drift measures whether there is an overall shift in the pulse pressure data during acquisition, such as numerical drift caused by poor sensor contact. Smaller drift indicates stronger data stability. Signal-to-noise ratio (SNR) measures the ratio of effective pulse signal to environmental and equipment noise; higher SNR indicates a higher proportion of effective signal and better data purity. These two indicators are quantified using algorithms, and then a weighted score is calculated based on preset weights. This initial confidence score for the pulse diagnosis data ranges from [0,1]. A higher score indicates more reliable pulse diagnosis data acquisition. The core function of the initial confidence score is to filter low-quality data and correct its initial probability allocation. When the initial confidence score of a certain modality of evidence source is lower than a preset threshold, its data acquisition quality is deemed substandard, and its corresponding initial probability allocation result needs to be subjected to maximum entropy fuzzification. The processing logic is: reduce the confidence of the evidence source in the specific syndrome, i.e., the probability of the specific syndrome in the initial probability allocation, while increasing the probability allocation for the TCM syndrome set Θ, which is now an unknown set, making the probability distribution more uniform and reducing the misleading effect of low-quality data on subsequent fusion results. Conversely, if the score exceeds the preset threshold, it indicates that the data quality is qualified, and its initial probability allocation result remains unchanged, directly entering the subsequent weighted fusion stage. It should be noted that the initial confidence levels of the audio diagnostic vector and the questioning vector can be obtained through the signal-to-noise ratio of the audio signal after noise reduction and the completeness and standardization of the questioning answers.

[0046] For example, a preset threshold of 0.6 is set. For tongue images, the clarity score is calculated using the Laplacian operator. Due to the dim ambient light during shooting, the image is blurry, and the initial confidence score is 0.45. For facial images, the lighting was sufficient and there was no shaking during shooting, resulting in a clarity score of 0.82, which is higher than the threshold of 0.6, indicating acceptable data quality. For corresponding pulse pressure data, the baseline drift quantization value is 0.1, indicating small drift, and the signal-to-noise ratio is 680:1. The overall calculated initial confidence score is 0.75, which is higher than the threshold of 0.6, indicating acceptable data quality. The audio signal has a high signal-to-noise ratio after noise reduction, scoring 0.8; the questioning answers are complete and standardized, scoring 0.9, both higher than the threshold of 0.6. The initial confidence scores of the data corresponding to the facial diagnosis vector, pulse diagnosis vector, voice diagnosis vector, and questioning vector are all ≥0.6, and their initial probability allocation results remain unchanged. They are then combined with the updated tongue diagnosis probability allocation results and entered into the subsequent weighted fusion stage. The original initial probability allocation result of the tongue diagnosis vector is... Tongue diagnosis showed Qi deficiency = 0.1 , After maximum entropy fuzzification, the probability of specific symptoms is reduced, the probability of the unknown set is increased, and the result is updated as follows: , , , The initial probability allocation results were updated to avoid misleading the diagnosis of Yin deficiency syndrome due to ambiguous tongue appearance.

[0047] In one embodiment, the initial probability allocation results are weighted and fused according to the consistency measurement principle and the uncertainty measurement principle to generate the confidence allocation results of each evidence source for each syndrome in the TCM syndrome set. Specifically, this includes: calculating the cosine similarity between each pair of evidence sources based on the consistency measurement principle; wherein, the cosine similarity is used to characterize the consistency of the two evidence sources in the direction of syndrome judgment, and the expression for the cosine similarity is: ; In the formula, For the first The source of evidence and the first Cosine similarity between evidence sources; For the first The initial probability assignment results of each source of evidence; For the first The initial probability assignment results of each source of evidence; The total number of syndromes in the TCM syndrome differentiation; The first in the collection of syndromes in traditional Chinese medicine Individual symptoms; for The source of evidence for the first The initial probability allocation results for each syndrome; For the first The source of evidence for the first The initial probability allocation results for each syndrome; The aggregate support is generated for each source of evidence based on cosine similarity; where aggregate support characterizes the total degree of support from other modal evidence sources for the current source of evidence, and the expression for aggregate support is: ; In the formula; For the first Aggregate support of each source of evidence; The modality number of the evidence source; For traversal index; Calculate the Dunk entropy for each source of evidence. Generate corrected weights based on the aggregated support, Dunk entropy, and initial confidence score of each source of evidence. Then, based on these corrected weights, generate the confidence assignment results for each source of evidence to each syndrome in the TCM syndrome set. The expressions for Dunk entropy and corrected weights are as follows: ; In the formula, For the first Deng's entropy of one source of evidence; Collection of Traditional Chinese Medicine Syndromes A subset, each subset representing a combination of syndromes, which can be a single syndrome or a combination of multiple syndromes; The number of symptoms contained in the subset; For the first The initial probability assignment results of each source of evidence to subset A; ; In the formula, For the first Adjusted weights for each source of evidence; This is the normalization function; , , These are custom weighting coefficients; For the first Initial confidence scores for each source of evidence; For the first The pre-defined weights for each source of evidence in traditional Chinese medicine (TCM) clinical practice are set based on TCM clinical experience.

[0048] In this embodiment of the invention, the core of consistency measurement is to quantify the degree of fit between any two evidence sources in syndrome judgment using cosine similarity, ensuring that consensus is highlighted and conflicts are weakened during fusion. The calculation object is the initial probability allocation result of each evidence source, i.e., the BAP vector. Each vector dimension is equal to the total number of syndromes in the TCM syndrome set, and the vector elements correspond to the initial probability allocation result of each syndrome by the evidence source. In the expression of cosine similarity, the numerator is the dot product of the two BPA vectors, which measures directional consistency. If both pieces of evidence have high confidence in the same syndrome, the dot product value is large; the denominator is the product of the magnitudes of the two vectors, used for normalization to ensure that the similarity value is between [0,1]. The closer the similarity is to 1, the more consistent the syndrome judgment direction of the two evidence sources; the closer it is to 0, the more obvious the conflict. Aggregate support is the total degree of support of a single evidence source by all other evidence sources. By traversing all other evidence sources, the cosine similarity of the current evidence source with theirs is summed to obtain the aggregate support, which is a comprehensive quantification of evidence consistency. In this embodiment of the invention, the number of evidence source modalities =5, then the range of aggregate support is [0,4]. The higher the score, the more consistent the conclusions of this evidence source are with other diagnostic methods, the stronger its credibility, and the higher its weight should be given in subsequent fusion; conversely, a low score indicates that there are many conflicts in this evidence source, and its influence needs to be reduced. Deng's entropy is used to quantify the uncertainty of a single evidence source, covering both random uncertainty and fuzzy uncertainty, and can be adapted to the characteristics of traditional Chinese medicine: "clear single symptoms, complex concurrent symptoms." Its core logic is: a single syndrome... When =1, the denominator =1, small entropy value, clear evidence; further evidence When =2, the denominator =3, entropy increases, evidence becomes ambiguous; TCM syndrome set, i.e., unknown set. When =6, the denominator =63, the entropy value is the highest, indicating the greatest uncertainty. The lower the entropy value, the more informational the evidence, and the higher the subsequent weight. The revised weights integrate four dimensions: data quality, evidence consistency, uncertainty, and clinical experience, ensuring that the fusion logic is scientific and aligned with the realities of TCM diagnosis and treatment. The initial weights are standardized using a normalization function to obtain the final revised weights. Finally, the initial probability allocation results of each evidence source are multiplied by the corresponding revised weights to obtain the confidence level allocation result of each evidence source for each syndrome, completing the weighted fusion.

[0049] In one embodiment, the confidence level allocation results for each syndrome in the TCM syndrome set are generated based on the modified weights, specifically including: The initial probability allocation result for each evidence source to each subset is obtained one by one. The initial probability allocation results for subsets with conflicting symptom elements are then corrected to obtain the corrected probability allocation result for each evidence source. The expression for the correction function is as follows: ; In the formula, This is the corrected initial probability assignment result for the i-th evidence source to subset A; For the first Adjusted weights for each source of evidence; For the first The initial probability assignment results of each source of evidence to subset A; Collection of Traditional Chinese Medicine Syndromes A subset, each subset representing a combination of syndromes, which can be a single syndrome or a combination of multiple syndromes; For the revised i-th source of evidence to the TCM syndrome set Trust level; The modified probability allocation results are subjected to orthogonal sum operations a preset number of times using Dempster's synthesis rules to generate confidence allocation results for each syndrome in the TCM syndrome set from each evidence source; where the preset number of operations is k-1, and k is the modality number of the evidence source.

[0050] In this embodiment of the invention, evidence sources with conflicting syndrome elements are discounted to avoid low-weight, highly conflicting evidence interfering with the final result. The discounting logic is based on a preset discounting function. Instead of directly eliminating conflicting evidence, it weakens its influence by adjusting the weights while retaining some valid information. The initial probability allocation result for each evidence source to all subsets of the TCM syndrome set is obtained one by one. For evidence sources with conflicting syndrome elements, such as tongue appearance indicating heat syndrome and pulse diagnosis indicating cold syndrome, the probability is adjusted according to the formula... Calculate the corrected probability: the smaller the correction weight, the lower the confidence level for the specific subset after correction. Also, according to the formula... The confidence level for the TCM syndrome set (i.e., the unknown set) is calculated, and the confidence levels of conflicting parts are transferred to the unknown set to ensure that the sum of probabilities of all subsets after correction is 1. After the corrected probability allocation results are generated, orthogonal summation is performed using Dempster's composition rule to integrate the confidence levels of all evidence sources and generate the final syndrome confidence allocation results. The core of orthogonal summation is to highlight the consensus parts of each evidence source and weaken the conflict parts to ensure the objectivity and reliability of the results. The number of operations is 4. Each operation calculates a new confidence allocation by summing the products of the intersection confidence levels of the two evidence sources for the corrected probability allocation results, gradually integrating all evidence sources. Finally, after k-1 orthogonal summation operations, the comprehensive confidence level of each syndrome is obtained. This confidence level is the confidence allocation result of each evidence source for each syndrome in the TCM syndrome set. The higher the confidence level, the stronger the fit between the corresponding syndrome and the user's health status. Then, the final generated confidence allocation results and multimodal evidence sources are input into the large language model. The large language model is built on an LLM neural network and is deeply fine-tuned through a professional dataset in the field of traditional Chinese medicine that includes clinical guidelines, expert consensus, and structured four diagnostic cases. It has the ability to make diagnostic thinking and reasoning that fits the clinical practice of traditional Chinese medicine. Preferably, the system will automatically select one of the dual processing engines to generate clinical judgment results and personalized treatment suggestions for users based on the degree of conflict of the confidence allocation results. (1) If the evidence is highly consistent, such as the confidence of a single syndrome being significantly higher than that of other syndromes, the ordinary engine mode is activated, the core expert sub-network is activated, and the conclusion is quickly generated by relying on the local knowledge base for retrieval; (2) If there are high conflict factors, such as the presence of mixed cold and heat syndromes, the deep thinking mode is activated, the multi-expert sub-network is dynamically activated, and multi-step complex logical reasoning is performed. For the input data received, the large language model will first summarize the user's current health status based on the syndrome with higher confidence allocation results, then analyze potential health risks, such as the possibility that long-term neglect of Yin deficiency may lead to insufficient liver Yin, and finally generate personalized treatment suggestions covering dimensions such as diet, daily life, exercise, emotions, and tea. To avoid model illusion, the generated preliminary report needs to be reviewed by experts. The modification traces produced after the review will form structured feedback data. The model will be optimized through reinforcement learning from human feedback (RLHF) technology, forming a closed loop of generation-review-optimization to ensure the safety and authority of medical advice.

[0051] In this embodiment of the invention, a TCM diagnosis and treatment decision-making device based on multimodal data is also provided, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the above-mentioned TCM diagnosis and treatment decision-making method based on multimodal data.

[0052] In this embodiment of the invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein the computer program controls the device where the computer-readable storage medium is located to execute the above-described TCM diagnosis and treatment decision-making method based on multimodal data when it is running.

[0053] For example, a computer program can be divided into one or more modules, one or more of which are stored in memory and executed by a processor to carry out the present invention. One or more modules can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a TCM diagnostic and treatment decision-making device based on multimodal data.

[0054] Traditional Chinese medicine (TCM) diagnostic and treatment decision-making devices based on multimodal data can be computing devices such as desktop computers, laptops, handheld computers, and cloud servers. These devices may include, but are not limited to, processors, memory, and displays. Those skilled in the art will understand that the above-mentioned components are merely examples of TCM diagnostic and treatment decision-making devices based on multimodal data and do not constitute a limitation on such devices. The device may include more or fewer components, or a combination of certain components, or different components. For example, a TCM diagnostic and treatment decision-making device based on multimodal data may also include input / output devices, network access devices, buses, etc.

[0055] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the multimodal data-based TCM diagnostic and treatment decision-making equipment, connecting all parts of the equipment through various interfaces and lines.

[0056] The memory can be used to store computer programs and / or modules. The processor, by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory, realizes various functions of the TCM diagnosis and treatment decision-making device based on multimodal data. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function (such as sound playback function, text conversion function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, text message data, etc.). In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0057] In this invention, the module for TCM diagnosis and treatment decision-making based on multimodal data, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. Those skilled in the art can understand and implement this invention without any inventive effort.

[0058] This invention provides a TCM diagnostic and treatment decision-making method based on multimodal data. Based on an asymmetric array sensor and dynamic gain adjustment strategy of a pulse diagnosis pen, it accurately collects the spatiotemporal pressure characteristics of the radial artery, generating pulse diagnosis vectors containing temporal and spatial dimensions, overcoming the pain points of traditional pulse diagnosis's subjectivity and incomplete data. Simultaneously, it standardizes the collection of tongue, facial, audio, and oral history data, extracting their core features to generate multidimensional feature vectors, comprehensively covering the four diagnostic methods of TCM, aligning with the overall diagnostic principle of integrating the four methods, and providing high-quality data support for subsequent treatment. Multimodal evidence sources are mapped to a unified TCM syndrome set to generate initial probability allocation results. Then, weighted fusion is performed using consistency and uncertainty measurement principles, effectively handling conflicts and uncertainties in heterogeneous data, improving the objectivity and reliability of syndrome judgment, and overcoming the diagnostic bias problems caused by traditional single-modal analysis or simple weighted fusion. Finally, the confidence allocation results and multimodal evidence sources are input into a large model trained on a professional TCM dataset. Its dual processing engine can adapt to different complex cases, generating personalized treatment suggestions. By deeply integrating traditional Chinese medicine diagnostic and treatment thinking with artificial intelligence and sensor technology, users can obtain professional-level TCM health assessments through mobile terminals, improving the accuracy and reliability of online diagnosis and treatment.

[0059] Example 2 See Figure 2 , Figure 2 This is a schematic diagram of a module of a traditional Chinese medicine intelligent auxiliary diagnosis and treatment system provided in one embodiment of the present invention. The embodiment of the present invention includes: The pulse diagnosis pen 201 and the host computer 202 are configured; the output end of the pulse diagnosis pen is connected to the input end of the host computer, and the host computer is equipped with intelligent programs and a large language model. The pulse pen is used to collect the user's pulse pressure data and transmit it to the host computer. The host computer is used to execute the TCM diagnosis and treatment decision-making method based on multimodal data as described in Example 1; wherein, the intelligent program is used to collect the user's tongue image, facial image, audio signal and diagnosis answer.

[0060] See Figure 3 , Figure 3This is a physical diagram of a TCM intelligent auxiliary diagnosis and treatment system provided in one embodiment of the present invention. In this embodiment, the pulse diagnosis pen is a dedicated pulse diagnosis data acquisition hardware for the system. Its core function is to objectively and accurately capture the pressure data of the user's radial artery and transmit it to the host computer. Its hardware design is adapted to the TCM "Three Parts and Nine Pulse Diagnosis Method," equipped with 24 high-precision capacitive pressure sensors arranged in a 5×5-1 asymmetric array, which can comprehensively cover the lateral and longitudinal pulsation range of the radial artery's cun, guan, and chi segments. Simultaneously, it has a built-in dynamic gain adjustment module that can automatically switch between three pulse-taking states (superficial, middle, and deep) and corresponding gain modes based on the acquired average static pressure, ensuring clear and distortion-free pressure signals under different pressure intensities. During the acquisition process, the pulse diagnosis pen synchronously records the ADC value and timestamp of each sensor at a 100Hz sampling rate, transmitting the raw data to the host computer in real time, laying the foundation for subsequent feature extraction. The host computer is typically a smartphone, tablet, or desktop computer. Its core function is to run intelligent programs and large language models, executing a complete set of TCM diagnostic and treatment decision-making methods based on multimodal data, completing the entire process from multi-source data acquisition, processing, and fusion to the generation of diagnostic and treatment recommendations. The host computer receives pulse pressure data transmitted by the pulse diagnosis pen through a communication module, and simultaneously relies on intelligent programs to expand the acquisition of data from the three diagnostic methods of observation, auscultation, and inquiry. Then, through built-in algorithms and large models, it completes data fusion and syndrome differentiation analysis, and finally outputs clinical judgment results and personalized diagnostic and treatment recommendations.

[0061] See Figure 4 , Figure 4 This is a schematic diagram of the homepage of an intelligent program provided in one embodiment of the present invention. In this embodiment, the intelligent program is a dedicated software module deployed in a host computer. Its core responsibility is to supplement the collection of data from the three diagnostic methods (diagnosis, palpation, and intestinal diagnosis) in addition to pulse diagnosis, thereby achieving data integrity for the integrated application of the four diagnostic methods in Traditional Chinese Medicine. Specifically, the program completes data collection through hardware interfaces of the host computer, such as cameras, microphones, and touchscreens: it collects images of the user's tongue and face through the camera, collects audio signals of the user reading samples of the five-tone characters through the microphone, presents a body composition scale conforming to the standards of the China Association of Traditional Chinese Medicine through an interactive interface, guides the user to provide feedback on their diagnostic answers, and then converts these data into standardized multi-dimensional feature vectors, which are integrated with the pulse diagnosis vectors to form a multimodal evidence source. The large language model, specifically a deeply fine-tuned LLM model for the field of Traditional Chinese Medicine, is deployed in the host computer and serves as the core algorithm carrier for realizing intelligent diagnosis and treatment. The model receives and processes multimodal evidence sources, as well as the syndrome confidence allocation results after weighted fusion through consistency and uncertainty measures. Combined with professional training data such as TCM clinical guidelines and structured four diagnostic cases, the model performs syndrome differentiation reasoning through a dual processing engine to generate clinical judgments and personalized suggestions. At the same time, it relies on a closed loop of generation-review-feedback optimization to ensure the safety and professionalism of the suggestions.

[0062] For example, a user uses a smartphone equipped with this system and a matching pulse diagnosis pen to complete a traditional Chinese medicine (TCM) health assessment. The user opens the TCM smart diagnosis and treatment APP on their phone, activates the pulse diagnosis function, and the APP automatically searches for and connects to the pulse diagnosis pen via Bluetooth, completing device pairing and communication initialization. At this point, the pulse diagnosis pen enters the data acquisition state. The user places the pulse diagnosis pen against the radial artery area on the inside of their wrist, adjusting its position so that the sensor array covers the cun, guan, and chi segments. The pulse diagnosis pen automatically starts data acquisition, recording the pressure ADC values ​​and timestamps of 24 sensors at a sampling rate of 100Hz. The acquisition lasts for 30 seconds, during which the gain mode is dynamically adjusted based on the average static pressure to ensure signal quality. After acquisition, the pulse diagnosis pen transmits 3000 sets of raw data to the mobile APP in real time. The mobile APP guides the user to complete the acquisition of the remaining three diagnostic data: ① taking pictures of the tongue and face using the rear camera; ② guiding the user to read aloud samples of the five tones of the Chinese pentatonic scale (Gong, Shang, Jiao, Zhi, Yu), collecting audio through the microphone and reducing noise; ③ presenting a body weight scale through a pop-up window, where the user answers questions such as "Do you feel cold?" and "Sleep quality?" The APP automatically maps the answers into Boolean vectors and degree vectors. The app utilizes built-in algorithms to extract temporal and spatial features from pulse diagnosis data to generate pulse vectors, and extracts core features from tongue, facial, audio, and consultation data to generate multi-dimensional feature vectors. All vectors are mapped to a set of traditional Chinese medicine syndromes to generate initial probability allocation results. Weighted fusion is then performed using cosine similarity and Dönbrunn entropy to obtain the confidence allocation results for each syndrome. The app inputs the multimodal evidence sources and confidence results into a large language model. Since there is no significant conflict in the evidence, the model activates normal mode, quickly generating clinical judgments and personalized suggestions. The preliminary report is submitted to TCM experts for review. Once approved, it is pushed to the user's app, where the user can view the complete health report. Throughout the process, the pulse pen accurately collects pulse data, the host computer integrates multi-source data through intelligent programs, and the algorithm and large model perform syndrome differentiation analysis. The collaborative operation of all components not only restores the essence of the four diagnostic methods in traditional Chinese medicine but also standardizes and automates the diagnosis and treatment process through technological means.

[0063] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0064] This invention provides a TCM intelligent auxiliary diagnosis and treatment system. Through a collaborative architecture of a pulse diagnosis pen and a host computer, it achieves a deep integration of traditional TCM diagnosis and treatment with intelligent technology. The pulse diagnosis pen, with its professional hardware design, accurately collects pulse pressure data, overcoming the shortcomings of traditional pulse diagnosis, which is subjective and difficult to quantify, thus providing objective evidence for diagnosis. The host computer, through intelligent programs, completes multimodal data including tongue appearance, facial appearance, audio, and medical history, fully covering the four diagnostic dimensions of TCM and conforming to the core principle of integrated diagnosis and treatment. Simultaneously, the host computer executes a dedicated diagnosis and treatment algorithm, combined with a deeply fine-tuned large language model, to complete data fusion and diagnostic reasoning. This effectively handles conflicts between multi-source data, reduces uncertainty, and generates personalized treatment suggestions. The efficient collaboration of all components preserves the essence of TCM diagnosis and treatment while improving accuracy and efficiency through standardized processes, balancing medical professionalism with ease of use.

[0065] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several improvements and substitutions without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A TCM diagnosis and treatment decision-making method based on multimodal data, characterized in that, Suitable for TCM intelligent auxiliary diagnosis and treatment systems that include pulse diagnosis pens, including: The pulse pressure data of the user is collected by the pulse pen, and the temporal and spatial features of the pulse pressure data are extracted to generate a pulse vector; wherein, the pulse pen adopts a dynamic gain adjustment strategy and arranges several sensors in an asymmetric array structure to collect the pulse pressure data. The system collects the user's tongue image, facial image, audio signal, and consultation answers, extracts core features, and generates a multi-dimensional feature vector; wherein, the multi-dimensional feature vector includes tongue diagnosis vector, facial diagnosis vector, audio diagnosis vector, and consultation vector; Using the pulse diagnosis vector and the multidimensional feature vector as multimodal evidence sources, the multimodal evidence sources are mapped to a pre-constructed set of TCM syndromes, and the initial probability allocation results of each evidence source for each syndrome in the set of TCM syndromes are generated one by one; wherein, the set of TCM syndromes contains several syndromes. The initial probability allocation results are weighted and fused according to the consistency measurement principle and the uncertainty measurement principle to generate the confidence allocation results of each evidence source for each syndrome in the TCM syndrome set. The multimodal evidence sources and the confidence allocation results are then input into a pre-constructed large language model so that the large language model can generate clinical judgment results and personalized diagnosis and treatment suggestions for the user. The large language model is based on an LLM neural network model and is trained using a professional dataset in the TCM field that includes clinical guidelines and structured four diagnostic cases.

2. The TCM diagnosis and treatment decision-making method based on multimodal data as described in claim 1, characterized in that, The pulse pen employs a dynamic gain adjustment strategy and uses several sensors arranged in an asymmetric array structure to collect the pulse pressure data, specifically including: The pulse diagnosis pen includes several pressure sensors; wherein the sensors are arranged in an array structure, and the central area of ​​the array is left empty, and static pressure calibration is performed based on the central area of ​​the array to align with the radial artery pulsation center point. The pressure sensors arranged in the array structure cover the lateral and longitudinal fluctuation range of the radial artery to collect the user's pulse pressure data. Based on the threshold range of the average static pressure monitoring value of the pressure sensor in the array structure, the pulse diagnosis pen switches between pulse-taking state and gain mode; wherein, the pulse-taking state includes floating pulse, deep pulse and middle pulse, and the gain mode includes high gain mode, low gain mode and default gain mode.

3. The TCM diagnosis and treatment decision-making method based on multimodal data as described in claim 1, characterized in that, The process of collecting pulse pressure data from users using a pulse pen and extracting the temporal and spatial features of the pulse pressure data to generate a pulse vector specifically includes: The pulse pressure data of the user is continuously collected based on a preset sampling rate; wherein, the pulse pressure data includes the ADC value and collection timestamp of each sensor. The ADC value is mapped to the actual pressure value and the pressure value is reconstructed into a pressure heatmap based on an interpolation algorithm; wherein, the pressure heatmap is used to characterize the diameter, length and pulsation intensity of the radial artery; The pressure heatmaps are sorted according to the acquisition timestamps, and the sorted pressure heatmaps are converted into a four-dimensional spacetime tensor to obtain the pulse diagnosis vector; wherein, the expression of the four-dimensional spacetime tensor is: ； In the formula, This represents the number of pulse pressure data samples collected in a single session. The time step represents the number of consecutively acquired time frames. These are characteristic channels, including AC channels and DC channels; This represents the vertical pixel dimension of the pressure heatmap, corresponding to the vertical distribution of the radial artery. This represents the horizontal pixel dimension of the pressure heatmap, corresponding to the horizontal distribution of the radial artery.

4. The TCM diagnosis and treatment decision-making method based on multimodal data as described in claim 1, characterized in that, The process of collecting the user's tongue image, facial image, audio signal, and consultation answers, extracting core features, and generating a multi-dimensional feature vector specifically includes: The tongue image is color-corrected and the color-corrected tongue region is extracted. The color and texture features of the tongue region are extracted using the LAB color model to obtain the tongue diagnosis vector. The facial reflection areas of the facial image are identified, and the gloss and color deviation of the facial reflection areas are extracted using the LAB color model to obtain the facial diagnosis vector. Extract the MFCC features of the audio signal, construct a voiceprint feature matrix based on the MFCC features, and use the denoised voiceprint feature matrix as the voice diagnosis vector; wherein, the audio signal is the sound data of the user reading a preset five-tone character sample; The consultation answers are mapped to Boolean vectors or degree vectors to obtain the consultation vector; wherein, the consultation answers are the user's responses to several body mass scales that conform to the standards of the China Association of Traditional Chinese Medicine.

5. The TCM diagnosis and treatment decision-making method based on multimodal data as described in claim 1, characterized in that, Before weighting and fusing the initial probability allocation results according to the consistency measurement principle and the uncertainty measurement principle, the method further includes updating the initial probability allocation results based on the initial confidence scores of the multimodal evidence sources. Specifically: The sharpness scores of the tongue image and the facial image are calculated using a preset operator, and the sharpness scores are used as the initial confidence scores of the images. The initial confidence score of the pulse pressure data is calculated based on the baseline drift and signal-to-noise ratio of the pulse pressure data. The initial probability allocation results corresponding to modal evidence sources with initial confidence scores below a preset threshold are subjected to maximum entropy fuzzification to obtain updated initial probability allocation results; wherein, the initial confidence score is used to characterize the data acquisition quality.

6. The TCM diagnosis and treatment decision-making method based on multimodal data as described in claim 5, characterized in that, The step of weighting and fusing the initial probability allocation results according to the consistency measurement principle and the uncertainty measurement principle to generate the confidence allocation results of each evidence source for each syndrome in the TCM syndrome set specifically includes: Based on the consistency measurement principle, the cosine similarity between each pair of evidence sources is calculated one by one; wherein, the cosine similarity is used to characterize the consistency between the two evidence sources in the direction of symptom judgment, and the expression for the cosine similarity is: ； In the formula, For the first The source of evidence and the first Cosine similarity between evidence sources; For the first The initial probability assignment results of each source of evidence; For the first The initial probability assignment results of each source of evidence; The total number of syndromes in the TCM syndrome differentiation; The first in the collection of syndromes in traditional Chinese medicine Individual symptoms; for The source of evidence for the first The initial probability allocation results for each syndrome; For the first The source of evidence for the first The initial probability allocation results for each syndrome; The aggregated support is generated for each source of evidence based on the cosine similarity; wherein the aggregated support characterizes the total degree to which the current source of evidence is supported by other modal sources of evidence, and the expression for the aggregated support is: ； In the formula; For the first Aggregate support of each source of evidence; The modality number of the evidence source; For traversal index; Calculate the Dunk entropy for each source of evidence, generate corrected weights based on the aggregated support, Dunk entropy, and initial confidence score of each source of evidence, and generate confidence assignment results for each syndrome in the TCM syndrome set based on the corrected weights; wherein, the expressions for the Dunk entropy and the corrected weights are: ； In the formula, For the first Deng's entropy of one source of evidence; Collection of Traditional Chinese Medicine Syndromes A subset, each subset representing a combination of syndromes, which can be a single syndrome or a combination of multiple syndromes; The number of symptoms contained in the subset; For the first The initial probability assignment results of each source of evidence to subset A; ； In the formula, For the first Adjusted weights for each source of evidence; This is the normalization function; , , These are custom weighting coefficients; For the first Initial confidence scores for each source of evidence; For the first The pre-defined weights for each source of evidence in traditional Chinese medicine (TCM) clinical practice are set based on TCM clinical experience.

7. The TCM diagnosis and treatment decision-making method based on multimodal data as described in claim 6, characterized in that, The process of generating confidence assignment results for each syndrome in the TCM syndrome set based on the modified weights specifically includes: The initial probability allocation result for each evidence source to each subset is obtained one by one. The initial probability allocation results for subsets with conflicting symptom elements are then corrected to obtain the corrected probability allocation result for each evidence source. The expression for the correction function is as follows: ； In the formula, This is the corrected initial probability assignment result for the i-th evidence source to subset A; For the first Adjusted weights for each source of evidence; For the first The initial probability assignment results of each source of evidence to subset A; Collection of Traditional Chinese Medicine Syndromes A subset, each subset representing a combination of syndromes, which can be a single syndrome or a combination of multiple syndromes; For the revised i-th source of evidence to the TCM syndrome set Trust level; The modified probability allocation results are subjected to a preset number of orthogonal sum operations using Dempster's synthesis rules to generate confidence allocation results for each syndrome in the TCM syndrome set from each evidence source; wherein the preset number of operations is k-1, and k is the modality number of the evidence source.

8. A Traditional Chinese Medicine Intelligent Auxiliary Diagnosis and Treatment System, characterized in that: include: A pulse diagnosis pen and a host computer; wherein the output end of the pulse diagnosis pen is connected to the input end of the host computer, and the host computer deploys intelligent programs and a large language model; The pulse diagnosis pen is used to collect the user's pulse pressure data and transmit it to the host computer. The host computer is used to execute the TCM diagnosis and treatment decision-making method based on multimodal data as described in any one of claims 1 to 7; wherein, the intelligent program is used to collect the user's tongue image, facial image, audio signal and diagnosis answer.

9. A terminal device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the TCM diagnosis and treatment decision method based on multimodal data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the TCM diagnosis and treatment decision-making method based on multimodal data as described in any one of claims 1 to 7.

Images