Deep learning-based auxiliary diagnosis method, device and equipment for four examinations in traditional Chinese medicine

Abstract

The present application relates to the technical field of traditional Chinese medicine auxiliary diagnosis, and discloses a traditional Chinese medicine four-examination auxiliary diagnosis method, device and equipment based on deep learning, which realizes automatic detection and classification of tongue diagnosis, face diagnosis and pulse diagnosis through deep learning technology, converts traditional Chinese medicine three-examination information into structured data, combines a traditional Chinese medicine knowledge graph and a large language model, constructs a four-examination combined reasoning mechanism, integrates three-examination structured data and patient complaint symptoms, and intelligently generates pathogenesis analysis, conditioning suggestions and medical treatment prompts; meanwhile, a platform integrating data collection, intelligent diagnosis and report visualization output functions is built, the digitization and standardization of traditional Chinese medicine four-examination are realized, the subjectivity of traditional Chinese medicine diagnosis is effectively reduced, the diagnosis efficiency and accuracy are improved, convenient and reliable technical support is provided for traditional Chinese medicine auxiliary diagnosis, and the present application has a wide practical application prospect.

Description

Technical Field

[0001] This invention relates to the field of auxiliary diagnostic technology in traditional Chinese medicine, and in particular to a method, device, and equipment for auxiliary diagnosis in traditional Chinese medicine based on deep learning and the four diagnostic methods. Background Technology

[0002] The four diagnostic methods of Traditional Chinese Medicine (TCM) are the core means of syndrome differentiation and treatment. However, traditional TCM diagnosis relies on the personal experience of physicians and suffers from the following technical pain points: insufficient feature quantification; diagnostic information such as tongue appearance, facial complexion, and pulse is mostly unstructured data, lacking objective quantitative standards, resulting in highly subjective diagnostic results; fragmented multi-source data; heterogeneous data types for tongue diagnosis, facial diagnosis, pulse diagnosis, and medical history inquiry, such as image data for tongue and facial diagnosis, waveform data for pulse diagnosis, and text data for medical history inquiry, making it difficult to achieve unified reasoning and fusion analysis; scarcity of high-quality datasets; a lack of publicly available, structured datasets of tongue and pulse appearance covering various TCM subcategories, hindering the training and optimization of intelligent diagnostic models; and an imbalance between diagnostic efficiency and professionalism, with traditional diagnostic processes being complex and reliant on the experience of senior physicians, making it difficult to meet the needs of large-scale and standardized diagnosis and treatment. Existing intelligent TCM diagnostic technologies mostly focus on the automated identification of single diagnostic methods, lacking deep integration of information from the four diagnostic methods and a systematic decision-making mechanism, resulting in insufficient diagnostic accuracy and practicality, making it difficult to widely apply in clinical scenarios. Summary of the Invention

[0003] This invention provides a method, device, and equipment for auxiliary diagnosis in traditional Chinese medicine based on deep learning, which solves the technical problems of traditional Chinese medicine four diagnostic methods relying on the subjective experience of physicians, difficulty in quantifying diagnostic features, and difficulty in fusion of multimodal information.

[0004] The first aspect of this invention provides a deep learning-based method for assisting diagnosis using the four diagnostic methods of traditional Chinese medicine, comprising the following steps: Step 1: Construct a three-diagnosis model, a TCM Prompt template, and a TCM knowledge graph, and optimize the large language model using LoRA technology; Step 2: Obtain the patient's tongue, facial, and pulse data, and generate structured three-diagnosis results through the three-diagnosis model. The three-diagnosis results include tongue diagnosis results, facial diagnosis results, and pulse diagnosis results. Step 3: Obtain the patient's consultation data, and integrate the consultation data, the results of the three diagnostic methods, and the TCM knowledge graph through a retrieval enhancement generation strategy to extract similar text slices; Step 4: Guide the optimized large language model based on the TCM Prompt template, and generate structured diagnosis and treatment results by combining the similar text slices.

[0005] A second aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described deep learning-based auxiliary diagnostic method for traditional Chinese medicine's four diagnostic methods.

[0006] A third aspect of the present invention provides a deep learning-based auxiliary diagnostic device for traditional Chinese medicine (TCM) diagnosis, comprising a computer-readable storage medium and a processor, wherein the processor executes a computer program on the computer-readable storage medium to implement the steps of the deep learning-based auxiliary diagnostic method for TCM diagnosis.

[0007] A fourth aspect of this invention provides a deep learning-based auxiliary diagnostic device for the four diagnostic methods of traditional Chinese medicine, including a construction module, a three-diagnosis module, a fusion reasoning module, and a result generation module. The building module is used to construct a three-diagnosis model, a TCM Prompt template, and a TCM knowledge graph, and optimizes the large language model through LoRA technology. The three-diagnosis module is used to acquire the patient's tongue, facial, and pulse data, and generate structured three-diagnosis results through the three-diagnosis model. The three-diagnosis results include tongue diagnosis results, facial diagnosis results, and pulse diagnosis results. The fusion reasoning module is used to acquire patients' consultation data and integrate the consultation data, three diagnostic results and TCM knowledge graph through a retrieval enhancement generation strategy to extract similar text slices; The result generation module is used to guide the optimized large language model based on the TCM Prompt template and generate structured diagnosis and treatment results by combining the similar text slices.

[0008] The beneficial effects of this invention are as follows: This invention provides a method, device, and equipment for auxiliary diagnosis in Traditional Chinese Medicine based on deep learning, which adopts the core technical path of "structured perception of the three diagnostic methods - decision-making based on the four diagnostic methods - implementation on a Web platform", and has the following beneficial effects: (1) For the three types of unstructured data, tongue diagnosis, face diagnosis and pulse diagnosis, we designed exclusive modeling methods. Tongue diagnosis is based on a lightweight model to realize the automatic detection and classification of six features such as tongue spirit and tongue color. Face diagnosis is based on a multi-color space distance matching algorithm to analyze the color features of five major areas of the face. Pulse diagnosis is based on a multi-layer perceptron model to model multiple quantifiable indicators such as pulse rate and pulse rhythm, and complete the structured extraction of information from the three diagnoses.

[0009] (2) A fusion mechanism for the four diagnostic methods was constructed. The large language model was fine-tuned through long-distance radio (LoRA) technology, combined with the TCM knowledge graph and retrieval enhancement generation (RAG) strategy, and matched with the TCM-specific prompt word (Prompt) framework to realize the unified reasoning of the structured data of the three diagnostic methods and the consultation text, generating structured diagnosis and treatment plans such as pathogenesis analysis, conditioning suggestions and medical treatment prompts, and ensuring the professionalism of the output through multi-model cross-validation.

[0010] (3) A Web platform was built based on the Flask framework, integrating data collection, model reasoning, diagnostic report visualization and download functions, supporting users to independently complete tongue diagnosis, face diagnosis and pulse diagnosis data collection and consultation interaction.

[0011] To make the above-mentioned objects, features and advantages of the invention more apparent and understandable, preferred embodiments of the invention are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating a deep learning-based auxiliary diagnostic method for traditional Chinese medicine's four diagnostic methods, provided in one embodiment. Figure 2 This is a schematic diagram of the structure of a tongue image classification model in one embodiment; Figure 3 This is a schematic diagram of a traditional Chinese medicine knowledge graph as an example. Figure 4 This is a schematic diagram of the four diagnostic methods integration link in one embodiment; Figure 5 This is a schematic diagram of the structure of a deep learning-based auxiliary diagnostic device for the four diagnostic methods of traditional Chinese medicine, provided in one embodiment. Figure 6 This is a schematic diagram of the structure of a TCM four diagnostic auxiliary diagnostic device based on deep learning, provided in one embodiment. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0015] It should be noted that, unless otherwise specified, the various features in the embodiments of this invention can be combined with each other, all of which are within the protection scope of this invention. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this invention do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0016] Figure 1 This is a flowchart illustrating a deep learning-based auxiliary diagnostic method for traditional Chinese medicine's four diagnostic methods, as provided in Example 1. Figure 1 As shown, it includes the following steps: Step 1: Construct a three-diagnosis model, a TCM Prompt template, and a TCM knowledge graph, and optimize the large language model using LoRA technology; Step 2: Obtain the patient's tongue, facial, and pulse data, and generate structured three-diagnosis results through the three-diagnosis model. The three-diagnosis results include tongue diagnosis results, facial diagnosis results, and pulse diagnosis results. Step 3: Obtain the patient's consultation data, and integrate the consultation data, the results of the three diagnostic methods, and the TCM knowledge graph through a retrieval enhancement generation strategy to extract similar text slices; Step 4: Guide the optimized large language model based on the TCM Prompt template, and generate structured diagnosis and treatment results by combining the similar text slices.

[0017] The above embodiments provide a deep learning-based auxiliary diagnostic method for the four diagnostic methods of Traditional Chinese Medicine (TCM). Through deep learning technology, it automates the detection and classification of tongue diagnosis, facial diagnosis, and pulse diagnosis, transforming traditional TCM diagnostic information into structured data. Combining TCM knowledge graphs and large language models, it constructs a synergistic reasoning mechanism for the four diagnostic methods, integrating the structured data from the three methods with the patient's chief symptoms to intelligently generate pathogenesis analysis, treatment suggestions, and medical advice. Simultaneously, it builds a platform integrating data acquisition, intelligent diagnosis, and visualized report output, achieving the digitization and standardization of the four diagnostic methods of TCM. This effectively reduces the subjectivity of traditional TCM diagnosis, improves diagnostic efficiency and accuracy, and provides convenient and reliable technical support for TCM auxiliary diagnosis, with broad practical application prospects.

[0018] The following specific embodiments will be used to describe each step of the above method in detail.

[0019] First, this invention combines modern image processing technology with deep learning methods to construct a classification model adapted to diagnostic tasks such as tongue diagnosis, facial diagnosis, and pulse diagnosis. The overall approach relies on the concept of multimodal data fusion to fully explore the TCM diagnostic information contained in tongue images, facial images, and pulse waveforms, thereby realizing the digital and intelligent transformation of the traditional TCM diagnostic process. By converting the multi-source unstructured data collected by the perception layer into standardized structured information, it provides a solid data foundation and technical support for the subsequent development of intelligent consultation processes and personalized diagnostic recommendations.

[0020] As those skilled in the art know, the tongue is connected to the internal organs of the human body through the meridian system, and different parts of the tongue have relatively fixed correspondences with the internal organs. In one specific embodiment, to address the problems of traditional tongue diagnosis relying on experience and insufficient feature quantification, this invention combines the TCM theory of tongue organ zoning with lightweight target detection technology to construct a complete technical chain from tongue image data acquisition and multi-dimensional feature modeling to intelligent classification, thereby achieving automated identification of key diagnostic elements such as tongue spirit, tongue color, tongue shape, tongue state, tongue coating texture, and tongue coating color.

[0021] Specifically, in a preferred embodiment, generating tongue diagnosis results includes the following steps: S101: Collect historical tongue image samples of different tongue image categories, label them, and establish training and testing sets.

[0022] For example, to address the technical problem of the lack of publicly available, structured, and high-quality tongue image datasets covering various subcategories of TCM tongue diagnosis in existing technologies, which hinders the effective construction of deep learning-based intelligent tongue diagnosis systems, this invention employs a keyword-driven web crawling approach. It combines the TCM tongue image classification system to construct a tongue image dataset covering multiple tongue features. The specific implementation process is as follows: First, based on the tongue image classification system (tongue spirit, tongue color, tongue shape, tongue posture, tongue coating, and tongue coating color) and the specific descriptions of tongue images in TCM theory, a combination of tongue image keywords is constructed. Then, a Python-based image acquisition and crawling system is designed and implemented. A query URL is constructed using the Bing image search engine, automatically parsing the search results page, extracting image links, and downloading high-quality initial tongue image samples corresponding to different tongue image categories. To improve the medical relevance of the images and the effectiveness of tongue region detection, a pre-trained model is introduced to perform tongue detection on the crawled image samples. Only candidate samples with high detection confidence and meeting medical image standards are retained, avoiding irrelevant or low-quality images from affecting model training. After multiple rounds of screening, approximately 40-100 medically relevant candidate images were retained for each tongue image category for subsequent model training and validation. During the construction process, images were renamed, formatted, and the dataset was automatically divided to ensure that the data structure was standardized, the labels were consistent, and the model could easily process the data. The final result was a tongue image dataset that covers the main tongue image categories, has good medical reference value, and is experimentally reproducible.

[0023] In a preferred embodiment, preset data augmentation techniques, including Mosaic and MixUp, can also be used to enrich the alternative tongue image samples under different lighting conditions and / or different shooting angles, so as to improve the robustness of the model under complex tongue image conditions such as changes in lighting and differences in shooting angles.

[0024] S102, construct a tongue image classification model, and train the model parameters of the tongue image classification model based on the training set and the test set. The training process adopts a full-precision floating-point calculation method until the preset convergence target is reached, thereby ensuring the training stability of the classification model in the scenario of small sample datasets, effectively avoiding the model convergence problem caused by gradient instability, and further ensuring the consistency and reliability of training results.

[0025] For example, to meet the need for accurate multi-attribute detection in tongue diagnosis images, the tongue image classification model of this invention is built on the PyTorch framework, employing an improved Feature Pyramid Network (FPN) and a unified task architecture design to achieve efficient collaboration between target detection and instance segmentation. When performing tongue image target detection and classification, to avoid information interference between different tasks and adapt to the multi-attribute classification requirements of tongue diagnosis, this invention has specifically optimized the model detection head. Specifically, it independently sets the bounding box regression branch and classification branch of the detection head based on a decoupled structure, enhancing the specialized learning capabilities of each branch. The bounding box regression branch uses the CIoU loss function to improve the fitting accuracy of the predicted box in terms of position, size, and shape, while the classification branch uses the BCE loss function to support independent modeling of each category, enabling parallel output of the patient's tongue diagnosis results in dimensions such as tongue spirit, tongue color, tongue shape, tongue state, tongue coating texture, and tongue coating color, achieving simultaneous prediction across multiple diagnostic dimensions.

[0026] Simultaneously, a lightweight neck feature enhancement module is constructed between the backbone network and the detection head to further ensure the accuracy of the classification results. For example, the structure of the tongue image classification model in one embodiment is as follows: Figure 2 As shown.

[0027] Finally, S103 is executed to collect the patient's current tongue image and call the trained tongue image classification model to generate tongue diagnosis results, including tongue spirit, tongue color, tongue shape, tongue posture, tongue coating texture, and tongue coating color.

[0028] In another embodiment, a facial feature matching method based on multi-color space distance calculation is used to obtain the facial diagnosis results. As is known to those skilled in the art, different areas of the face have a clear correspondence with internal organs; for example, the forehead corresponds to the heart, the left cheek to the liver, the right cheek to the lungs, the nose to the spleen and stomach, and the jaw to the kidneys. Each area becomes a window reflecting the function of its corresponding organ. Furthermore, Traditional Chinese Medicine classifies facial color into five colors: green, red, yellow, white, and black, each associated with different symptoms, such as red indicating heat, yellow indicating dampness, and white indicating deficiency. In practical facial diagnosis applications, based on the spatial correspondence between facial areas and internal organs, facial segmentation detection technology is used to accurately divide core areas such as the forehead, left and right cheeks, nose, and jaw, facilitating targeted observation. Considering that bluish complexion is easily affected by light and skin tone in clinical practice and is difficult to identify, it is often simplified into four color categories: red, yellow, white, and black. By analyzing the color depth, brightness, and distribution characteristics of each area, such as redness on the forehead inferring heart heat, yellowing of the nose as related to spleen and stomach dampness, and paleness of the right cheek as suggesting lung deficiency, the inference of the health status of the corresponding organs and the analysis of disease diagnosis can be achieved.

[0029] In a preferred embodiment, generating a face-to-face consultation result includes the following steps: S201: Acquire the patient's facial image, perform diagnostic region segmentation on the facial image, and label the segmentation results. Specifically, after performing face detection on the input facial image, extract facial feature points using the 68-point facial keypoint localization model provided by dlib. This localization method requires no additional training and can significantly reduce computational overhead and improve processing efficiency while ensuring high localization accuracy. Then, based on traditional Chinese medicine facial diagnosis theory, five core diagnostic regions are identified: forehead, nose, left cheek, right cheek, and mandible. Combining the extracted keypoint location information, the center point of each diagnostic region and the corresponding region of interest coordinate range are calculated. Subsequently, images of the above-mentioned diagnostic regions are extracted from the original facial image for subsequent processing, and each diagnostic region is labeled with Chinese characters in the original facial image to enhance the visualization effect.

[0030] S202, the face image is converted to several color spaces, the average pixel value of each channel of each diagnostic region in each color space is calculated, and a color feature vector corresponding to each color space is established.

[0031] Specifically, firstly, four types of baseline facial color images are loaded, corresponding to the common red, white, black, and yellow complexions in Traditional Chinese Medicine theory. Then, the images of each diagnostic area and each baseline facial color image are converted to three different color spaces: HSV, Lab, and YCrCb. The HSV color space includes three channels: H (Hue), ranging from 0° to 360° (0° for red, 120° for green, and 240° for blue); S (Saturation), ranging from 0% to 100% (0% for grayscale and 100% for pure color); and V (Lightness), ranging from 0% to 100% (0% for black and 100% for maximum brightness). The expression for converting an image from RGB space to HSV space can be written as: .

[0032] The expression for converting from RGB space to Lab space can be written as: .

[0033] The expression for converting an image from RGB space to YCrCb space can be written as: , This allows the images of each diagnostic region of a face image to be converted to the three color spaces mentioned above.

[0034] Then, for the five facial diagnostic regions—forehead, left cheek, right cheek, nose, and jaw—the average pixel value of each channel in four color spaces (RGB, Lab, HSV, and YCrCb) is calculated for each region. This average value is used to represent the color space coordinates of the corresponding diagnostic region and form a multidimensional color feature vector. For example, the average pixel values ​​in the four color spaces for the forehead region of a face image are shown in the table below: RGB (R,G,B) Lab (L,a,b) HSV (H,S,V) YCrCb(Y,Cr,Cb) forehead (210, 180, 160) (75, 10, 15) (20, 0.2, 0.8) (180, 140, 130) S203, query the pixel reference values ​​of each channel for each diagnostic region when it presents different reference surface colors in each color space, and calculate the Euclidean distance between each diagnostic region and different reference surface colors in each color space based on the color feature vector. The Euclidean distance formula is: , in The average value of each channel in the color space of the input image. Let C be the channel mean of the corresponding region in the reference image, and C be the number of channels in the color space. Then, sum the total distances between each diagnostic region and different reference surface colors. The target reference surface color with the smallest total distance is taken as the current surface color of the corresponding diagnostic region, and the diagnostic results are output.

[0035] For example, when querying the forehead with a base color of red, the base pixel values ​​for each channel in each color space are as follows: RGB (R,G,B) Lab (L,a,b) HSV (H,S,V) YCrCb(Y,Cr,Cb) forehead (220, 180, 160) (78, 12, 16) (18, 0.25, 0.82) (185, 142, 128) The Euclidean distances between the forehead and the reddish skin color in the four color spaces (RGB, Lab, HSV, YCrCb) are calculated as follows:

[0036] Therefore, the total distance between the forehead area and the reddish complexion .

[0037] Similarly, the total distances between the forehead area and white, black, and yellow skin tones can be calculated to be 79, 125, and 160 respectively, thus determining that the skin tone of the forehead area is red.

[0038] As those skilled in the art know, the formation of pulse is influenced by a variety of factors. Traditionally, it is analyzed from four aspects: "position, number, shape, and momentum". Modern research further refines the core components of pulse into eight characteristics: pulse position, pulse rate, pulse length, pulse width, pulse strength, pulse rhythm, smoothness, and tension. These eight characteristics cover multiple dimensions such as the spatial location, frequency rhythm, and morphological tension of the pulse. Systematically mastering these indicators helps to better establish pulse diagnosis models.

[0039] For example, in one embodiment, generating pulse diagnosis results includes the following steps: S301, the user's first pulse signal is acquired by a single-point pulse sensor, and a first feature parameter is extracted based on the first pulse signal. The first feature parameter includes pulse strength, pulse rhythm, beat rate, tension and / or fluency. S302, the user's second pulse signal is acquired through a sensor array, and a second feature parameter is extracted based on the second pulse signal. The second feature parameter includes pulse position, pulse length and / or pulse width. S303, invoke a preset first neural network model and generate a first pulse classification result based on the first feature parameters; at the same time, invoke a preset second neural network model and generate a second pulse classification result based on the second feature parameters. S304, dynamically assign weights and fuse the first pulse classification result and the second pulse classification result to generate the target pulse classification result and confidence level.

[0040] Here, pulse position refers to the depth of the pulse on the skin surface, perceived by varying finger pressure; pulse rate is the frequency of the pulse per unit time, reflecting the speed of blood and qi circulation in the body; pulse length describes the distribution range of the pulse along the axis (three parts), reflecting the degree of blood and qi flow; pulse width indicates the transverse thickness of the pulse channel under the fingers, related to blood volume and vascular tension; pulse force reflects the strength of the pulsation, indicating the abundance or deficiency of blood and qi and the excess or deficiency of pathogenic factors; pulse rhythm is used to determine whether the pulse rhythm is uniform and whether there are abnormal intervals; smoothness assesses whether the pulse is smooth and round or sluggish and obstructed, reflecting the state of blood and qi circulation; tension characterizes the tension and elasticity of the pulse, related to the tightness of the pulse channel. Through comprehensive analysis of the above eight characteristics, the seven basic types of pulse (i.e., normal pulse and "six-category pulse") can be determined relatively accurately, thus providing a reliable basis for pulse diagnosis.

[0041] The above preferred embodiment analyzes the user's pulse strength, rhythm, rate, tension, and / or fluency by collecting pulse signals from a single-point sensor, and analyzes the user's pulse position, length, and / or width by collecting pulse signals from a sensor array. It also selects or fuses the pulse classification results obtained from both methods, taking into account factors such as pulse diagnosis objectives, real-time signal quality, and sensor characteristics. This not only allows for the separate training of sample data from the single-point sensor and the sensor array to establish neural network models, improving the model's generalization ability, but also provides flexible hardware settings and a diagnostic process that considers both the rhythm and morphological information of the pulse, thus improving the accuracy and applicability of the pulse diagnosis classification results.

[0042] In a preferred embodiment, since pathological abnormality data from the six principles of TCM pulse diagnosis are relatively scarce, a large model-based generation technique can be used to address the data imbalance problem, thereby generating labeled pathological data that conforms to the characteristics of TCM pulse diagnosis. The generated data follows the same format as the sensor-acquired data; for example, the sampling frequency is 50Hz, each data segment lasts 30 seconds, and the amplitude is normalized to a range of 0 to 127 to ensure consistency with the actual sensor data acquisition standards. In this way, the generated pathological pulse data effectively supplements and expands the dataset, providing more diverse and standardized samples for subsequent model training.

[0043] As those skilled in the art know, pulse position corresponds to the depth of pulse. Different pressure levels of multi-layered signals are required to simulate different pressure intensities such as "light pressure, medium pressure, and heavy pressure" to palpate layer by layer to determine the superficial, medium, and deep pulse characteristics. Pulse length is the longitudinal range of the pulse. Sensors need to be deployed simultaneously at the cun, guan, and chi positions to sense the distribution range of the pulse along the radial artery, thereby comprehensively assessing the pulse length. Pulse width is the width of the vascular pulsation (i.e., the degree of radial expansion), which can be captured by deploying a pressure-sensitive array sensor in a transverse multi-channel configuration. The specific acquisition and feature analysis methods are described in detail in existing technical documents and will not be repeated here.

[0044] For example, the following describes the extraction method of the first characteristic parameters such as pulse strength, pulse rhythm, beat rate, tension and / or fluency. However, the present invention is not limited to these calculation methods. Other similar methods for obtaining the above characteristic parameters are within the protection scope of the present invention.

[0045] The pulse rate, or beat count, can be obtained by peak detection in the PPG waveform. Common methods include peak localization algorithms based on sliding windows. By counting the number of beats per unit time, the pulse rate per minute can be converted into a pulse rate per minute. Therefore, pulse rate per minute is chosen to represent the change in the pulse rate. The calculation formula is as follows: , in, Pulse rate per minute For time window, for Detected within seconds Each pulse peak.

[0046] Pulse rhythm, or the uniformity of the rhythm, can be determined by extracting the RR interval between two adjacent pulse peaks, constructing an RR sequence, and calculating its standard deviation (std) and root mean square square (RMSSD). Fourier analysis or time-domain variability analysis of the RR interval can reveal whether premature beats or missed beats exist, thus determining whether the rhythm is stable. Since std only counts the overall dispersion and is not sensitive enough to "rhythm abrupt changes," while RMSSD precisely reflects the frequency and degree of interpulse interval abrupt changes, it can be expressed as: , in, This represents the time interval between two adjacent pulse peaks. The sequence length is given.

[0047] Pulse strength, or the intensity of the pulse, can be estimated using the amplitude characteristics of the waveform. A higher waveform amplitude generally indicates a stronger pulse, and vice versa. Therefore, this invention uses the average amplitude of the pulse peak to characterize pulse strength, which can be expressed as: , in, Indicates the detected number Each pulse peak.

[0048] Tension, or the pulsational tension, can be indirectly estimated through indicators such as the slope of the rising phase and peak sharpness of the waveform. Fluidity, or the smoothness and continuity of the pulsation, can be analyzed by examining the continuity and slope changes of the rising and falling phases of the waveform. Since the rising phase is commonly used in medicine to assess blood outflow velocity, the falling phase, being more influenced by factors such as vascular elasticity, is less stable than the rising phase. Therefore, this study chose to use the rising slope of the rising phase to characterize changes in tension and fluidity, which can be expressed as: , in, for The amplitude value of the rising branch waveform at time 1. for The amplitude value of the rising branch waveform corresponding to the given moment.

[0049] The first and second feature parameters of the user's current pulse can be obtained by analyzing the above methods. Then, the preset first neural network model and the preset second neural network model are called to obtain the classification results. The classification results include each possible pulse type and its probability distribution.

[0050] For example, both the preset first neural network model and the preset second neural network model can use a lightweight feedforward neural network (MLP) as the classifier. This has a small number of parameters, fast inference speed, and can be seamlessly deployed on mobile devices, embedded devices, and cloud servers. Specifically, the classifier of the first neural network model sequentially consists of an input layer, two hidden layers, and an output layer. The input layer receives the first feature parameters and passes them to the first hidden layer. The first hidden layer consists of a fully connected layer with 4 inputs and 64 outputs, a ReLU activation function, and a Dropout layer with a dropout rate of 20%, used for initial feature extraction and overfitting suppression. The second hidden layer consists of a fully connected layer with 64 inputs and 32 outputs, a ReLU activation function, and a Dropout layer with a dropout rate of 20%, used for deepening feature learning. Finally, the data is transmitted to the output layer, processed by a fully connected layer with 32 inputs and 7 outputs, and then converted into probability distributions of various pulse types by a Softmax activation function. The second neural network model can also use a lightweight feedforward neural network (MLP), which will not be elaborated further here. In other embodiments, the second neural network model can also use a lightweight CNN model, all of which are within the scope of protection of this application.

[0051] Finally, the two pulse classification results are selected or merged. In a preferred embodiment, S304 generates the target pulse classification result and confidence level as follows: S3041, the first pulse classification result and the second pulse classification result are filtered to retain similar pulses, remove conflicting pulses and / or insignificant pulses, and normalize the classification probabilities. Here, similar pulses refer to pulses that appear in both the first and second pulse classification results, insignificant pulses refer to pulses with probabilities lower than a preset value, and conflicting pulses refer to pulses that are obviously contradictory, which can be obtained by querying a preset table.

[0052] S3042, Align the preprocessed first pulse classification results and second pulse classification results, and arrange them in the order of common pulses and unique pulses; S3043, obtain the calculation weights corresponding to the first pulse pattern classification result and the second pulse pattern classification result. For common pulse patterns, directly include them in the target pulse pattern classification result, and perform weighted calculation based on the calculation weights to generate the corresponding classification probability. For unique pulse patterns, determine whether the corresponding classification probability is greater than a preset threshold. If so, include it in the target pulse pattern classification result and generate the corresponding classification probability according to the calculated weight.

[0053] In a preferred embodiment, the weights can be automatically adjusted based on factors such as pulse diagnosis goals (key targets for doctors), real-time signal quality, and sensor characteristics. For example, the weight of the array sensor can be actively increased when a floating pulse is suspected, the weight of a channel can be automatically reduced when the suspected data signal is unstable, and a higher initial weight can be given to a specific user when the sensor has more information, collects more accurate data, or is more stable.

[0054] For example, the calculation weights corresponding to the first pulse diagnosis result and the second pulse diagnosis result are obtained as follows: Initial weights are generated based on the confidence scores of the first and second pulse pattern classification results. For example, a higher confidence score indicates that the model has greater confidence in the output results, and the initial weights are also larger. Similarly, a mapping table can be built based on historical data to obtain the initial weights by looking up the table.

[0055] Then, correction coefficients are generated based on the pulse diagnosis target, the first data quality parameter of the single-point pulse sensor, and / or the second data quality parameter of the sensor array. Since the single-point pulse sensor acquires a time-series pulse wave signal at a single location, its quality parameters reflect the periodic stability and waveform fidelity of the signal. Therefore, factors such as signal amplitude shift, signal-to-noise ratio, waveform regularity, and the tightness of the sensor's contact with the skin can be considered. That is, by acquiring these parameter values ​​and normalizing them, the first comprehensive quality index of the single-point pulse sensor is obtained. Similarly, for the sensor array, the second data quality parameters include the stability of pressure distribution, the uniformity of pressure, and the consistency of pulse waveforms among related sensors, etc. The second comprehensive quality index is also calculated, thus obtaining a correction coefficient that comprehensively considers all factors.

[0056] The initial weights are optimized based on the correction coefficients to generate calculated weights for the first pulse classification result and the second pulse classification result. In this way, through the quantification of the above-mentioned multidimensional quality parameters, the system can sense the sensor status and adaptively adjust the confidence level, avoiding the output of unreliable fusion results when the signal deteriorates, and significantly improving the robustness and clinical applicability of the system.

[0057] After obtaining the results of tongue diagnosis, facial diagnosis, and pulse diagnosis, a technical chain was constructed based on a large language model, consisting of "three-diagnosis data - structured fusion - knowledge graph-enhanced reasoning - intelligent question generation," such as... Figure 4As shown, this paper first integrates TCM knowledge graphs with structured tongue, facial, and pulse diagnosis features using Retrieval Enhancement Generation (RAG) technology to address the semantic fragmentation of multimodal information. Second, a TCM-specific Prompt template is designed to guide the Large Language Model (LLM) in generating treatment suggestions that conform to the principles of syndrome differentiation and treatment. Based on this, a scale based on multi-model evaluation is designed to enhance the professionalism and interpretability of assisted diagnosis. Finally, an intelligent online consultation interface supporting multi-turn interactions is developed, simultaneously generating visualized treatment reports for both doctors and patients, improving the practicality and user experience of the method and the corresponding system.

[0058] In a preferred embodiment, the large language model includes a locally deployed DeepSeek model and a cloud-based model accessed via the DeepSeek API platform. The DeepSeek model is built on advanced multimodal learning technology, integrating large-scale pre-training and multimodal knowledge graphs, exhibiting superior performance in natural language understanding, reasoning, and generation, particularly demonstrating excellent semantic association capabilities in domain-specific knowledge processing. Its model design integrates convolutional neural networks and recurrent neural networks, achieving efficient processing of multimodal input data through complex semantic embedding and knowledge-driven frameworks. The cloud-based model, accessed via the DeepSeek API platform, offers advantages such as fast response speed and strong processing capabilities, making it suitable for large-scale data analysis and complex reasoning scenarios. The locally deployed large language model is more suitable for applications with high data privacy requirements, enabling multi-round consultations and small-scale experiments while ensuring data security, and features zero additional call costs. The two types of models can be flexibly switched according to specific application needs, thus achieving a good balance between performance and security.

[0059] In a preferred embodiment, to improve the adaptability of the large language model in TCM intelligent diagnosis tasks, the parameters of the DeepSeek model are fine-tuned using LoRA technology. Specifically, a question-and-answer format dataset is first established, which contains a large number of TCM-related question-and-answer pairs, covering common TCM syndromes, treatment suggestions, prescription analysis, and other fields. 500-1000 high-quality question-and-answer pairs are randomly selected from this dataset as the training set.

[0060] Then, LoRA technology is used to replace some linear layer parameters of the basic DeepSeek model with low-rank matrix combinations, and the low-rank matrix combinations are trained using the training set to enhance the model's semantic understanding and generation capabilities for TCM terminology, symptom descriptions, and / or pulse features. LoRA is a low-rank adaptation method whose core idea is to freeze the pre-trained model parameters and introduce only a small number of trainable parameters to specific layers to adapt the model to new tasks. Compared with traditional full-scale fine-tuning methods, it significantly reduces training costs and memory consumption. Moreover, the fine-tuned large language model significantly improves the accuracy and practicality of TCM professional content while maintaining the original general language understanding capabilities. Furthermore, LoRA makes minimal changes to the original model structure, so the model's inference speed and deployment efficiency are almost unaffected.

[0061] In a preferred embodiment, the present invention constructs a traditional Chinese medicine knowledge graph, specifically as follows: A TCM knowledge graph dataset, TCM-KG-triples, was constructed, containing several triplet data, covering various entity and relation types such as "symptom-medicine", "syndrome-symptom", "medicine-dosage" and "medicine-property". These triplets together construct the structured knowledge foundation of the TCM field, covering rich key information such as medicinal properties, treatment mechanisms and clinical manifestations.

[0062] Then, the Neo4j graph database is used to model and visualize the triplet data, and the entities and their corresponding relationships are imported into the graph database using the Cypher query language, so that each entity node can be connected to other semantically related nodes through edges, such as... Figure 3 As shown, this intuitively presents the interaction relationships among various entities in the TCM knowledge graph, providing convenient support for subsequent semantic retrieval and relational reasoning.

[0063] In a preferred embodiment, when step 3 integrates and extracts similar text slices using a retrieval enhancement generation strategy, a vectorization method based on a Chinese embedded model is employed to capture deep semantic relationships between entities in order to achieve semantic retrieval functionality for entities in the TCM knowledge graph. This includes four steps: entity embedding representation, vector index construction, semantic similarity query, and metadata return. For example, as shown... Figure 4As shown, for important concept nodes such as "Chinese herbal medicine" and "syndrome" in the TCM knowledge graph, their core text attributes are extracted and encoded using a pre-trained semantic representation model to obtain normalized high-dimensional semantic vector representations. These semantic vector representations are stored as node attributes in a graph database. Based on this, a vector index structure for entities is constructed to support subsequent approximate vector retrieval processes. In the semantic query stage, user input, such as user information and user consultation data, is converted into semantic vectors and their similarity is calculated with the embedded vectors of already stored entities. For example, the cosine similarity between vectors is used to determine the semantic similarity between two sentences, thereby retrieving the set of entities with the most similar semantics, resulting in similar text slices.

[0064] Finally, the retrieved similar text slices are input into the Traditional Chinese Medicine (TCM) Prompt template. The TCM Prompt template integrates the information and guides the optimized large language model to obtain structured diagnostic results. An example Prompt semantic framework design is as follows: Task Requirements: Based on patient consultation data (gender, age, past medical history, chief complaint) and the three diagnostic methods (tongue diagnosis, facial diagnosis, and pulse diagnosis), generate structured treatment suggestions. The output format should be JSON, containing the following fields: Pathogenesis analysis; Recommendations include dietary advice, traditional Chinese medicine advice, and exercise advice. Medical Advice: Explains when further medical attention is recommended.

[0065] Example of patient information input: PatientInfo: { "Gender": "Female", Age: 35 "Medical_History": None Symptoms: "Frequent fatigue, irritability, insomnia, intestinal discomfort, and sluggishness." "Tongue_Diagnosis": "Tongue spirit: withered tongue, tongue color: pale white, tongue shape: old tongue, tongue state: flaccid and soft tongue, coating quality: thin coating, coating color: white coating", "Face_Diagnosis": "Left cheek: pale, right cheek: reddish, jaw: dark, nose: yellowish, forehead: reddish". "Pulse_Diagnosis": "Mean oxygen saturation: 85%, mean pulse rate: 72 bpm, mean perfusion index: 10%, pulse type: slow"} The keywords extracted from the TCM knowledge graph are: intestinal heat excess syndrome, sluggish reaction, cold excess and yang deficiency syndrome.

[0066] The following is an example of a structured output of diagnostic results: { "Pathogenesis": "Deficiency of both Qi and Blood, and deficiency of Spleen and Kidney Yang, leads to insufficient production of Qi and Blood, resulting in pale complexion and fatigue." "Recommendations": { "Diet": "Eat more foods that warm and nourish the spleen and kidneys, replenish qi and blood, such as red dates, astragalus, goji berries, longan, and millet porridge." "Herbal": "I suggest taking a modified Bazhen Tang (Eight Treasures Decoction) to regulate qi and blood, and improve symptoms such as fatigue." "Exercise": "It is recommended to engage in gentle exercises such as Tai Chi or Baduanjin for 30 minutes daily to help improve blood circulation." "Medical Advice": "If symptoms continue to worsen, or severe dizziness occurs, you should promptly visit a Traditional Chinese Medicine (TCM) internal medicine department for further diagnosis and treatment based on tongue and pulse diagnosis." As those skilled in the art know, due to the strong dependence on professional background and semantic ambiguity of TCM-assisted diagnostic tasks, the results of their generative tasks are often difficult to effectively judge using traditional automatic evaluation indicators. This is especially true when it involves dialectical reasoning, theoretical citations, and individualized treatment suggestions; the rationality and professionalism of the large model output need to be comprehensively judged in conjunction with the TCM knowledge system. In a preferred embodiment, this invention uses a method for evaluating the output quality of a large language model for TCM-assisted diagnosis to systematically evaluate the appropriateness and practical value of the diagnostic and treatment results generated by the large model for a specific prompt from multiple dimensions.

[0067] The evaluation steps are as follows: First, an assessment scale is constructed, which has five core dimensions. The first dimension is "Accuracy of Diagnosis," which refers to whether the answer is based on information from the four diagnostic methods of Traditional Chinese Medicine (TCM), combined with the chief complaint and physical signs, to reasonably determine the syndrome type, such as Qi deficiency, Yin deficiency, damp-heat, etc.; and whether it reflects the basic principle of "treatment based on syndrome differentiation." The second dimension is "Theoretical Consistency," which refers to whether the TCM theories cited in the answer are correct and rigorous, and whether there are any logical errors or misuse of terminology. The third dimension is "Reasonableness of Treatment Suggestions," which refers to whether the proposed treatment suggestions are consistent with the diagnosis conclusion and have a TCM basis, avoiding "clichés" and "universal suggestions." The fourth dimension is "Clinical Practicality," which refers to whether the answer is realistically feasible and can provide reference value for preliminary clinical judgment or subsequent treatment, avoiding vague and unrealistic statements. The fifth dimension is "Clarity and Professionalism of Expression," which refers to whether the expression is fluent, whether the terminology is used correctly, whether it conforms to the writing habits of TCM, and whether the content is too general or unprofessional. Each dimension employs a scoring system. The evaluation process references both traditional Chinese medicine academic standards and actual clinical needs, assessing the model's comprehensive capabilities in understanding user symptoms, applying traditional Chinese medicine theories, and generating intervention-value recommendations. During the evaluation process, multiple models are used to score the diagnostic and treatment results across the various dimensions of the assessment scale, generating cross-validation results to more intuitively reflect the overall quality of the large model's output.

[0068] For example, a preferred embodiment of the present invention further includes a result output step: a web platform is built based on the Flask framework to generate consultation data and three-stage diagnosis results from patient information collected through an interactive interface, and a corresponding visual diagnostic report is generated based on the consultation data, three-stage diagnosis results, reasoning process and treatment results, which is convenient for doctors and patients to download.

[0069] This invention focuses on information from the four diagnostic methods of Traditional Chinese Medicine (tongue, face, pulse, and questioning). By constructing a TCM-assisted diagnostic technology path that integrates perception, fusion, and decision-making, and by integrating a diagnostic system with multimodal perception and a large language model, it has achieved significant technical results and practical value. Firstly, a deep learning-based three-diagnosis perception modeling method was proposed, which effectively solved the technical pain point of difficulty in efficiently extracting unstructured data such as tongue appearance, facial color and pulse in TCM diagnosis. It uses a lightweight model to achieve accurate detection and classification of tongue appearance areas, uses a multi-color space fusion algorithm to accurately analyze facial skin color features, and uses a multilayer perceptron to complete pulse waveform modeling and classification. This achieves efficient structured extraction and standardized representation of tongue, facial and pulse information, which greatly improves the utilization efficiency and accuracy of TCM diagnostic data.

[0070] Secondly, an integrated TCM consultation and synergistic decision-making mechanism was designed to solve the technical problem of difficulty in unified reasoning of multi-source heterogeneous diagnostic information. A standardized TCM consultation process was built based on a large language model. The Prompt project and RAG retrieval enhancement strategy were combined to guide the model to jointly understand and reason about tongue diagnosis images, facial features, pulse data and text questions and answers. The TCM question and answer quality evaluation index designed in conjunction with the model enhances the credibility of auxiliary diagnosis from the perspective of content rationality and professionalism, and ensures the scientificity and reliability of the diagnostic results.

[0071] Third, a web platform based on the Flask framework is built, integrating full-process functions such as data acquisition, model inference, and diagnostic report output. It has excellent stability and scalability, providing solid engineering support for multimodal perception fusion in traditional Chinese medicine.

[0072] Experimental results show that the method proposed in this invention exhibits excellent classification performance and diagnostic rationality in TCM-assisted diagnostic tasks, effectively improving the efficiency and accuracy of TCM diagnosis and treatment. It provides a practical and feasible technical solution for the digital and intelligent upgrading of traditional TCM diagnosis and treatment, and has broad practical application prospects and practical value.

[0073] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0074] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described deep learning-based TCM four diagnostic auxiliary diagnostic method.

[0075] Figure 5 This is a schematic diagram of a deep learning-based auxiliary diagnostic device for the four diagnostic methods in Traditional Chinese Medicine, as provided in one embodiment. Figure 5 As shown, it includes a construction module 100, a three-diagnosis module 200, a fusion reasoning module 300, and a result generation module 400. The construction module 100 is used to construct a three-diagnosis model, a TCM Prompt template, and a TCM knowledge graph, and to optimize the large language model through LoRA technology. The three-diagnosis module 200 is used to acquire the patient's tongue, facial, and pulse data, and generate structured three-diagnosis results through the three-diagnosis model. The three-diagnosis results include tongue diagnosis results, facial diagnosis results, and pulse diagnosis results. The fusion reasoning module 300 is used to acquire the patient's consultation data and integrate the consultation data, the results of the three diagnostic methods, and the TCM knowledge graph through a retrieval enhancement generation strategy to extract similar text slices; The result generation module 400 is used to guide the optimized large language model based on the TCM Prompt template and generate structured diagnosis and treatment results by combining the similar text slices.

[0076] It should be noted that the foregoing explanation of the embodiment of the TCM four diagnostic methods based on deep learning also applies to the TCM four diagnostic devices based on deep learning in the above embodiments, and will not be repeated here.

[0077] This invention also provides a deep learning-based auxiliary diagnostic device for traditional Chinese medicine's four diagnostic methods, including a computer-readable storage medium and a processor. When the processor executes a computer program on the computer-readable storage medium, it implements the steps of the deep learning-based auxiliary diagnostic method for traditional Chinese medicine's four diagnostic methods described above.

[0078] Figure 6 This is a schematic diagram of the structure of a deep learning-based auxiliary diagnostic device for traditional Chinese medicine's four diagnostic methods, as provided in one embodiment. Figure 6 As shown, the deep learning-based TCM four-diagnostic auxiliary device 8 of this embodiment includes: a processor 80, a readable storage medium 81, and a computer program 82 stored in the readable storage medium 81 and executable on the processor 80. When the processor 80 executes the computer program 82, it implements the steps in the various method embodiments described above, for example... Figure 1 The steps shown. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module in the above-described device embodiments, for example... Figure 5 The functions of the module shown.

[0079] For example, the computer program 82 can be divided into one or more modules, which are stored in the readable storage medium 81 and executed by the processor 80 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 82 in the deep learning-based traditional Chinese medicine four diagnostic methods auxiliary diagnostic device 8.

[0080] The deep learning-based TCM four diagnostic methods auxiliary diagnostic device 8 may include, but is not limited to, a processor 80 and a readable storage medium 81. Those skilled in the art will understand that... Figure 6 This is merely an example of a deep learning-based auxiliary diagnostic device for the four diagnostic methods of traditional Chinese medicine (TCM), and does not constitute a limitation on the deep learning-based auxiliary diagnostic device for the four diagnostic methods of TCM. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the deep learning-based auxiliary diagnostic device for the four diagnostic methods of TCM may also include a power management module, a computing processing module, input / output devices, network access devices, buses, etc.

[0081] The present invention is not limited to the description in the specification and embodiments, and thus other advantages and modifications can be readily realized by those skilled in the art. Therefore, the present invention is not limited to the specific details, representative devices and illustrated examples shown and described herein without departing from the spirit and scope of the general concept as defined by the claims and their equivalents.

Claims

1. A deep learning-based auxiliary diagnostic method for traditional Chinese medicine's four diagnostic methods, characterized in that, Includes the following steps: Step 1: Construct a three-diagnosis model, a TCM Prompt template, and a TCM knowledge graph, and optimize the large language model using LoRA technology; Step 2: Obtain the patient's tongue, facial, and pulse data, and generate structured three-diagnosis results through the three-diagnosis model. The three-diagnosis results include tongue diagnosis results, facial diagnosis results, and pulse diagnosis results. Step 3: Obtain the patient's consultation data, and integrate the consultation data, the results of the three diagnostic methods, and the TCM knowledge graph through a retrieval enhancement generation strategy to extract similar text slices; Step 4: Guide the optimized large language model based on the TCM Prompt template, and generate structured diagnosis and treatment results by combining the similar text slices.

2. The TCM four diagnostic methods based on deep learning as described in claim 1, characterized in that, The large language model includes a locally deployed DeepSeek model and a cloud model accessed via the DeepSeek API platform. The optimization of the large language model using LoRA technology specifically involves: Establish a training set, which contains several TCM question-and-answer pairs, covering the fields of TCM syndrome types, treatment suggestions, and prescription analysis; LoRA is used to replace some linear layer parameters of the basic DeepSeek model with a combination of low-rank matrices, and the low-rank matrix combination is trained using the training set to enhance the model's semantic understanding and generation capabilities for TCM terminology, symptom descriptions, and / or pulse features.

3. The TCM four diagnostic methods based on deep learning as described in claim 1, characterized in that, The construction of the TCM knowledge graph specifically involves: A TCM knowledge graph dataset is constructed, which contains several triplet data, covering various entity and relation types such as "symptom-medicine", "syndrome-symptom", "medicine-dosage" and "medicine-property". The triplet data is modeled and visualized using the Neo4j graph database, and entities and their corresponding relationships are imported into the graph database using the Cypher query language, so that each entity node can be connected to other semantically related nodes through edges.

4. The TCM four diagnostic methods based on deep learning as described in claim 1, characterized in that, It also includes an evaluation step and a result output step. The evaluation step specifically involves: constructing an assessment scale and using multiple models to score the diagnosis and treatment results from several preset dimensions of the assessment scale, thereby generating cross-validation results. The specific steps for outputting the results are as follows: a web platform is built based on the Flask framework to generate consultation data and three-stage diagnosis results from patient information collected through the interactive interface, and a corresponding visual diagnostic report is generated based on the consultation data, three-stage diagnosis results, reasoning process and diagnosis results.

5. The TCM four diagnostic methods based on deep learning according to any one of claims 1-4, characterized in that, Generating tongue diagnosis results includes the following steps: Collect historical tongue images of different tongue types, label them, and establish training and testing sets; A tongue image classification model is constructed, and the model parameters of the tongue image classification model are trained based on the training set and the test set. The training process adopts a full-precision floating-point calculation method until the preset convergence target is reached. The system acquires the patient's current tongue image and uses a trained tongue image classification model to generate tongue diagnosis results. The tongue image classification model uses an improved feature pyramid network as the backbone network for multi-scale feature extraction. Based on a decoupled structure, it independently sets the bounding box regression branch and classification branch of the detection head. A lightweight neck feature enhancement module is constructed between the backbone network and the detection head. The bounding box regression branch uses the CIoU loss function, and the classification branch uses the BCE loss function to output the patient's tongue diagnosis results in multiple dimensions in parallel, including tongue spirit, tongue color, tongue shape, tongue state, tongue coating texture, and tongue coating color.

6. The TCM four diagnostic methods based on deep learning as described in claim 5, characterized in that, The specific steps for establishing the training and test sets are as follows: Construct tongue image keywords and collect initial tongue image samples corresponding to different tongue image categories based on a web crawler system; A pre-trained model is used to detect the tongue body in the initial tongue body image samples, and candidate tongue body image samples that meet preset conditions are obtained. Pre-defined data augmentation techniques were used to enrich the candidate tongue image samples under different lighting conditions and / or different shooting angles, and the tongue image categories were labeled to establish training and testing sets.

7. The TCM four diagnostic methods based on deep learning as described in claim 5, characterized in that, Generating consultation results includes the following steps: Acquire the patient's facial image, perform diagnostic region segmentation on the facial image, and label the segmentation results; The face image is converted to several color spaces, the average pixel value of each channel in each diagnostic region is calculated in each color space, and a color feature vector corresponding to each color space is established. The system queries the pixel reference values ​​of each channel for each diagnostic region when it presents different reference surface colors in each color space, calculates the distance between each diagnostic region and different reference surface colors in each color space based on the color feature vector, sums the distances between each diagnostic region and different reference surface colors to generate the total distance between each diagnostic region and different reference surface colors, takes the target reference surface color with the smallest total distance as the current surface color of the corresponding diagnostic region, and outputs the diagnostic results.

8. The TCM four diagnostic methods based on deep learning as described in claim 5, characterized in that, Generating pulse diagnosis results includes the following steps: The patient's first pulse signal is acquired by a single-point pulse sensor, and a first feature parameter is extracted based on the first pulse signal. The first feature parameter includes pulse strength, pulse rhythm, beat rate, tension and / or fluency. The patient's second pulse signal is acquired by a sensor array, and a second feature parameter is extracted based on the second pulse signal. The second feature parameter includes pulse position, pulse length and / or pulse width. A preset first neural network model is invoked, and a first pulse classification result is generated based on the first feature parameters. A preset second neural network model is also invoked, and a second pulse classification result is generated based on the second feature parameters. The first pulse classification result and the second pulse classification result are dynamically weighted and fused to generate the patient's target pulse classification result and confidence level.

9. A deep learning-based auxiliary diagnostic device for the four diagnostic methods in Traditional Chinese Medicine, characterized in that, It includes a construction module, a three-diagnosis module, a fusion reasoning module, and a result generation module. The building module is used to construct a three-diagnosis model, a TCM Prompt template, and a TCM knowledge graph, and optimizes the large language model through LoRA technology. The three-diagnosis module is used to acquire the patient's tongue, facial, and pulse data, and generate structured three-diagnosis results through the three-diagnosis model. The three-diagnosis results include tongue diagnosis results, facial diagnosis results, and pulse diagnosis results. The fusion reasoning module is used to acquire patients' consultation data and integrate the consultation data, three diagnostic results and TCM knowledge graph through a retrieval enhancement generation strategy to extract similar text slices; The result generation module is used to guide the optimized large language model based on the TCM Prompt template and generate structured diagnosis and treatment results by combining the similar text slices.

10. A deep learning-based auxiliary diagnostic device for the four diagnostic methods of traditional Chinese medicine, comprising a computer-readable storage medium and a processor, characterized in that, When the processor executes the computer program on the computer-readable storage medium, it implements the steps of the deep learning-based TCM four diagnostic auxiliary diagnostic method according to any one of claims 1-8.

Images