Traditional Chinese medicine constitution intelligent identification and dynamic interpretable conditioning scheme generation system

Abstract

The present application relates to the technical field of traditional Chinese medicine, and in particular to a traditional Chinese medicine constitution intelligent identification and dynamic interpretable conditioning scheme generation system, comprising a bimodal encoding module and a hybrid routing fusion module, a generative conditioning engine, and an online-offline closed-loop updating module. Through the tongue surface bimodal feature alignment and rule-neural network hybrid dynamic routing architecture, the present application effectively overcomes the semantic gap between heterogeneous modalities and the limitations of single black box decision. On the self-built 12000 real data set, the Top-1 accuracy rate of traditional Chinese medicine constitution identification is greatly increased from 72.3% of the prior art to 89.7%. At the same time, with the generative conditioning engine integrating strict medical safety verification and SHAP numerical contribution degree echo, not only is the dynamic prescription generation of multi-dimensional perception and thousands of methods for thousands of people realized, but also the "black box" trust crisis of AI diagnosis is completely broken, and the blind evaluation satisfaction of professional traditional Chinese medicine practitioners is significantly improved from 3.2 points to 4.6 points.

Description

Technical Field

[0001] This invention relates to the field of traditional Chinese medicine technology, specifically to a system for intelligent identification of TCM constitution and generation of dynamically interpretable conditioning plans. Background Technology

[0002] With the rapid development of artificial intelligence technology, its application in the field of medical auxiliary diagnosis is becoming increasingly widespread. Traditional Chinese medicine (TCM), as a treasure of Chinese medicine, contains rich medical wisdom in its four diagnostic methods: observation, auscultation and olfaction, inquiry, and palpation. In recent years, intelligent TCM constitution identification systems based on computer vision technology have emerged, providing strong support for the objectification and standardization of TCM. However, in actual clinical applications and commercial deployments, existing TCM constitution identification and conditioning recommendation systems still have the following significant technical shortcomings: First, modal information acquisition is limited, and there is a semantic gap in cross-modal feature fusion.

[0003] Existing tongue or facial diagnosis systems mostly employ single-modal convolutional neural networks (CNNs) for classification. For example, relying solely on tongue images to output discrete constitution labels fails to comprehensively utilize key observational information such as facial complexion and luster, resulting in a bottleneck in the system's accuracy in identifying complex constitutions (generally below 75%). Even when some systems attempt to incorporate multimodal data, they often use simple feature concatenation. Due to the significant heterogeneous semantic gap between the visual probability distribution features extracted from images and the semantic features based on text descriptions, this simple concatenation fails to achieve effective alignment of features in the same high-dimensional space, significantly reducing the gain effect of multimodal fusion.

[0004] Second, there is an irreconcilable contradiction between pure rule-driven and pure data-driven approaches, and there is a lack of adaptive decision-making mechanisms.

[0005] At the reasoning and decision-making level, existing auxiliary diagnostic systems typically fall into two extremes: either relying on a pure TCM expert rule base (such as a knowledge graph), which fails when encountering atypical symptom combinations not covered by ancient texts, exhibiting extremely poor generalization ability; or relying on a purely data-driven deep learning model, which is a typical "black box," lacking support from TCM principles in its output. Currently, the few systems that combine both approaches mostly employ fixed, static weighting strategies (such as 50% weighting each), failing to dynamically and intelligently allocate decision weights between "authoritative rules" and "data generalization" based on the typicality or atypicality of the patient's input characteristics.

[0006] Third, the treatment plan generation mechanism is rigid, and it lacks medical-grade security verification and interpretability when introducing large language models.

[0007] Existing treatment plans are mostly static, pre-set templates, unable to dynamically adjust based on a patient's specific clinical symptoms, age, gender profile, geographical location, and current season. This makes truly personalized treatment difficult to implement. While Large Language Models (LLMs) have demonstrated powerful generative capabilities in recent years, directly applying them to TCM diagnosis and treatment scenarios can easily create "illusions," leading to the output of irregular prescriptions or suggestions with significant medical safety risks. Furthermore, existing black-box models cannot intuitively explain to doctors and patients "why this constitution was diagnosed" (i.e., they lack feedback on the specific numerical contribution of input features to the output results), resulting in extremely low clinical credibility and patient acceptance.

[0008] Fourth, after the model is launched, it lacks a continuous learning loop based on data drift detection and rigorous statistical validation.

[0009] After deployment, medical AI models often experience changes in the distribution of user input data over time and with geographical expansion (i.e., "data drift"), leading to long-term stagnation or even continuous degradation in system performance. Existing online diagnostic systems typically only have a single front-end inference function and lack a mechanism to automatically quantify and detect the degree of data drift in the background and automatically trigger retraining. More critically, medical scenarios are extremely sensitive to updates. Existing systems lack a rigorous evaluation and interception mechanism based on statistical significance tests when iterating models, and lack engineered version control and rollback capabilities for configurations that are prone to fluctuations, such as large model prompts. This makes it extremely easy for inferior models or incorrect configurations to be released online, leading to medical accidents.

[0010] In summary, there is an urgent need in this field for a TCM constitution identification and dynamic interpretable conditioning scheme generation system that can overcome the cross-modal semantic gap, achieve dynamic adaptive fusion of rules and neural networks, ensure the absolute safety and interpretability of the generated scheme, and possess rigorous continuous evolution capabilities. Summary of the Invention

[0011] To achieve the above objectives, the present invention provides a system for intelligent identification of TCM constitution and generation of dynamically interpretable conditioning solutions, including: A bimodal coding module is used to extract the features of the tongue image and the facial image input by the user, and align the tongue image features and the facial image features to a shared embedding space to generate bimodal fusion features; The hybrid routing fusion module is used to receive the dual-modal fusion features and input them in parallel to the rule path based on the TCM knowledge graph and the neural network path based on deep learning; and to dynamically weight and fuse the output probabilities of the rule path and the neural network path through a learnable gating mechanism to obtain the final constitution identification result. The generative conditioning engine is used to construct prompt words based on the final constitution identification result and preset individual information, and call a large language model to generate structured conditioning suggestions; at the same time, it outputs interpretability information reflecting the contribution of each input feature to the constitution identification result. The online-offline closed-loop update module is used to collect image data and inference logs from online detection, trigger model retraining based on data drift detection results or newly added samples that have been manually reviewed, and automatically update and deploy the new model to the online system after it passes offline evaluation.

[0012] Furthermore, the bimodal coding module includes a tongue-image coding subnet; The tongue image encoding subnetwork adopts a hybrid network architecture of convolutional neural network and Transformer; wherein, the convolutional neural network is used to extract local visual features of the tongue image, and the Transformer is used to extract global dependency features of the tongue image based on a multi-head attention mechanism; the hybrid network outputs multi-dimensional soft label features, which represent the probability distribution of local attributes of the tongue image, specifically including the probability values ​​of tongue color, tongue shape, coating color and coating texture.

[0013] Furthermore, the bimodal coding module also includes a face coding subnet; The facial encoding subnet is constructed based on a visual large language model. The facial encoding subnet receives facial images and extracts prompt words by combining preset facial features, and outputs descriptive text containing facial complexion and luster attributes. Then, the descriptive text is parsed and transformed into facial structured features of preset dimensions through a preset mapping layer.

[0014] Furthermore, the bimodal coding module also includes a feature alignment layer; The feature alignment layer is used to concatenate the soft label features output by the tongue image encoding subnet with the facial structured features output by the face encoding subnet, and map them to a shared embedding space of the same dimension. During the model training phase of the feature alignment layer, the cosine contrast loss function is used to constrain and optimize the model in order to narrow the semantic distance between the tongue image features and facial features of the same user in the shared embedding space and reduce cross-modal differences. The system also includes an input preprocessing module; the input preprocessing module is configured to receive tongue and facial images of any size uploaded by the user, and then uniformly scale them to a preset standard resolution size before distributing them to the bimodal encoding module.

[0015] Furthermore, the hybrid routing fusion module includes a rule path submodule; The rule path submodule contains a pre-built TCM knowledge graph. The rule path submodule is configured to: extract local attribute labels from the dual-modal fusion features, perform association queries and matching in the traditional Chinese medicine knowledge graph, and calculate the first constitution probability distribution based on traditional Chinese medicine expert rules. .

[0016] Furthermore, the hybrid routing fusion module also includes a neural network path submodule; the neural network path submodule includes a multilayer perceptron and an attention mechanism network layer; the neural network path submodule is configured to: receive the dual-modal fusion features, perform nonlinear feature transformation through the multilayer perceptron, and dynamically assign importance to different feature dimensions using the attention mechanism to calculate a data-driven second physical probability distribution. ; The specific configuration of the learnable gating mechanism is as follows: The first physical probability distribution The second physical probability distribution Perform vector concatenation; The dynamic gating weight factor is calculated using a single-layer or multi-layer fully connected network with an activation function. Its calculation formula is expressed as: in This represents the learnable weight matrix of the network; Using the dynamic gating weight factor The outputs of the rule path and the neural network path are dynamically weighted and summed to obtain the final physical probability distribution, which is represented by the following formula: Wherein, the learnable weight matrix During the model training phase of the system, continuous updates and optimizations are performed using the backpropagation algorithm.

[0017] Furthermore, the generative conditioning engine includes a prompt word dynamic construction submodule and a large language model invocation submodule; The prompt word dynamic construction submodule maintains a preset prompt word template pool; the prompt word dynamic construction submodule is configured to: obtain the final constitution identification result, and simultaneously extract the current user's symptom information, user profile, current seasonal characteristics and geographical location information; fill the above multi-dimensional information into the corresponding prompt word template, and combine to generate structured individualized prompt words; The large language model calling submodule inputs the individualized prompt words into the pre-deployed large language model and constrains the large language model to output structured conditioning suggestions containing multiple dimensions in a preset data interaction format.

[0018] Furthermore, the generative conditioning engine also includes a suggested security verification layer; The suggested security verification layer is located at the output end of the large language model calling submodule; the suggested security verification layer is internally configured with a regular expression rule library and a medical sensitive word library; The security verification layer is configured to: before outputting the structured conditioning suggestions to the user's front end, use the regular expression rule library and the medical sensitive word library to match and filter the text content of the structured conditioning suggestions in order to intercept or correct illegal suggestions that pose medical safety risks; Explainable information is obtained and displayed in the following ways: The Shapley algorithm is used to calculate the specific numerical contribution of each local input feature extracted by the dual-modality coding module to the final physical fitness identification result output by the hybrid routing fusion module. The numerical contribution is displayed back to the user's front-end interface in the form of a visual heatmap or structured text to show the user the driving weight of specific tongue or facial features in diagnosing a specific constitution.

[0019] Furthermore, the online-offline closed-loop update module includes a data drift detection submodule; The data drift detection submodule is configured to: synchronize the image data features and inference result logs of the user's online detection input to the underlying data storage system in real time or periodically; and calculate the distribution difference between the newly added online input data features and the historical training set data features. The data drift detection is calculated using an algorithm that combines the Kolmogorov-Smirnov test with feature embedding distance. When the detected distribution difference exceeds a preset drift threshold, a retraining warning instruction is automatically triggered for the dual-modal coding module and the hybrid routing fusion module.

[0020] Furthermore, the online-offline closed-loop update module also includes an automated evaluation and interception submodule; The automated evaluation and interception submodule is configured to: upon receiving a retraining warning instruction or after manually reviewing and annotating new high-quality training samples, initiate incremental training or retraining of the model to generate candidate version models. The candidate version model is evaluated against the current online model on an offline reserved test set, comparing key performance indicators. Only when the key performance indicators of the candidate version model are superior to those of the current online model and pass the preset statistical significance test will the candidate version model be considered successful. When the candidate version model is deemed valid, it is allowed to proceed to the next step of the automatic update process. It also includes a canary release and configuration management submodule; the canary release and configuration management submodule manages model versions based on the model registry center; after the candidate version model is determined to be valid, it is automatically deployed to the canary environment for small-scale A / B testing; and after the test indicators are stable, a seamless traffic switch for full online service is performed. For the prompt word template pool in the generative conditioning engine, a code-based operation and maintenance mechanism is used for version control and update management to support rapid rollback of prompt word versions.

[0021] Beneficial effects This invention, through its pioneering tongue-surface dual-modal feature alignment and rule-neural network hybrid dynamic routing architecture, effectively overcomes the semantic gap between heterogeneous modalities and the limitations of single black-box decision-making. On a self-built real dataset of 12,000 cases, it significantly increases the Top-1 accuracy of TCM constitution identification from 72.3% to 89.7% of existing technologies. Simultaneously, with its generative conditioning engine integrating rigorous medical safety verification and SHAP numerical contribution feedback, this invention not only achieves multi-dimensional perception and personalized prescription generation, but also completely breaks down the "black box" trust crisis of AI consultation, significantly improving the blind evaluation satisfaction of professional TCM doctors from 3.2 to 4.6. More importantly, the system's built-in online-offline closed-loop module based on data drift quantification detection and statistical significance testing (p<0.05) endows the system with automated continuous evolution capabilities while adhering to the bottom line of medical safety. This fundamentally solves the engineering pain point of long-term performance degradation after the deployment of traditional medical models, possessing extremely high clinical promotion value and commercial technical barriers. Attached Figure Description

[0022] Figure 1 This is a system diagram of the TCM constitution intelligent identification and dynamic interpretable conditioning scheme generation system of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] The present invention will now be described in further detail with reference to the accompanying drawings: Example: As shown in the figure, this invention provides a system for intelligent identification of TCM constitution and generation of dynamically interpretable conditioning solutions, including: A bimodal coding module is used to extract the features of the tongue image and the facial image input by the user, and align the tongue image features and the facial image features to a shared embedding space to generate bimodal fusion features; The hybrid routing fusion module is used to receive the dual-modal fusion features and input them in parallel to the rule path based on the TCM knowledge graph and the neural network path based on deep learning; and to dynamically weight and fuse the output probabilities of the rule path and the neural network path through a learnable gating mechanism to obtain the final constitution identification result. The generative conditioning engine is used to construct prompt words based on the final constitution identification result and preset individual information, and call a large language model to generate structured conditioning suggestions; at the same time, it outputs interpretability information reflecting the contribution of each input feature to the constitution identification result. The online-offline closed-loop update module is used to collect image data and inference logs from online detection, trigger model retraining based on data drift detection results or newly added samples that have been manually reviewed, and automatically update and deploy the new model to the online system after it passes offline evaluation.

[0026] Furthermore, the specific operation procedure of the dual-modal coding module is as follows: The system first obtains the raw image data to be identified through the input preprocessing module. Specifically, the system receives the Uniform Resource Locator (URL) of the tongue image and the facial photo uploaded by the user through the front-end terminal. Considering that the images uploaded by users usually vary greatly in resolution and aspect ratio, and are of any size, the input preprocessing module first downloads the above images to local or cloud memory, and then uses bilinear interpolation or adaptive pooling algorithms to uniformly scale the tongue image and facial image to a preset standard resolution size. Pixel normalization is then performed to eliminate the interference of device acquisition differences on subsequent feature extraction.

[0027] The preprocessed tongue and facial images are input in parallel into different sub-networks of the dual-modal encoding module. For the tongue image, this embodiment employs a hybrid network combining a convolutional neural network (CNN) and a Transformer architecture as the tongue encoding sub-network. In the specific computation process, a CNN (e.g., a shallow ResNet structure) is first used to extract local visual feature maps of the tongue image; then, these feature maps are flattened, positionally encoded, and input into the Transformer module. The Transformer module captures the global contextual dependencies between local patches in the tongue image through a multi-head self-attention mechanism. The formula for calculating the single-head attention weights within the module is as follows: in, These represent the query matrix, key matrix, and value matrix obtained from the tongue image feature mapping, respectively. denoted as the dimension of the key vector. After the above hybrid network processing, the tongue image encoding subnetwork finally outputs an 8-dimensional soft-label feature vector through a fully connected layer and a Softmax activation function. Each dimension of this soft-label feature vector represents the probability of a specific local attribute of the tongue image. For example, the output result may be [pale white tongue color: 0.91, teeth marks on the tongue body: 0.88, yellow tongue coating: 0.12, greasy tongue coating: 0.05,...]. Compared with the traditional one-hot encoding, this soft-label representation can retain richer information on the uncertainty of constitution and is more in line with the fuzzy characteristics of TCM diagnosis.

[0028] Meanwhile, for feature extraction from facial images, this invention breaks through the limitations of traditional classification methods that rely solely on CNNs, innovatively introducing a Vision-LLM (Visual Large Language Model) into the facial encoding subnetwork. Specifically, the system inputs the preprocessed facial image and preset prompts (e.g., "Please describe the complexion and luster features of this face from the perspective of traditional Chinese medicine observation") into the Vision-LLM. The model first outputs descriptive text containing facial complexion and luster attributes (e.g., "Pale complexion, lacking luster"), and then maps this descriptive text into a fixed-dimensional structured facial feature vector through a pre-trained text embedding layer.

[0029] However, since the soft-label features of the tongue image belong to the visual probability distribution domain, while the structured facial features originate from the text semantic domain, there is a significant heterogeneous semantic gap between the two. To address this technical challenge, a feature alignment layer is further incorporated into the bimodal coding module. This feature alignment layer concatenates the aforementioned 8-dimensional soft-label features of the tongue image with the structured facial features, and then maps them to a 128-dimensional shared embedding space through a multilayer perceptron to obtain the aligned fused features.

[0030] To ensure alignment effectiveness, during the model training phase of this invention, the feature alignment layer employs a cosine contrast loss function to constrain and optimize network parameters. This loss function aims to shorten the distance between tongue and facial features of the same user (positive sample pair) in the 128-dimensional shared space, while simultaneously increasing the feature distance between different users (negative sample pairs). Its core calculation formula is represented as follows: in, The total number of samples within a batch. and These represent the first and second parts after mapping to the shared embedding space. The tongue feature vector and facial feature vector of each sample The cosine similarity between two vectors is represented by the following expression: For sample labels, when the tongue image and face belong to the same user ,otherwise , The preset margin is set to 0.5. Through continuous backpropagation optimization of this formula, the system can effectively overcome the semantic differences between modalities and output high-quality bimodal fusion features, thus laying a solid data foundation for downstream hybrid routing inference.

[0031] Furthermore, the specific operation process of the hybrid routing fusion module is as follows: After acquiring the high-dimensional fusion features and local soft-label features output by the dual-modal coding module, the data flows to the core decision-making center of this invention—the hybrid routing fusion module. Existing TCM-assisted diagnosis and treatment systems often face a dilemma: purely rule-based systems (such as expert systems) are too rigid and unable to handle complex combinations of atypical symptoms; while purely data-driven deep learning models are "black boxes," lacking the support of TCM theory and easily giving diagnostic results that contradict common medical sense. To completely solve this technical problem, this embodiment innovatively adopts a dual-path parallel and dynamically weighted hybrid routing network architecture.

[0032] Specifically, the hybrid routing fusion module includes a "rule path submodule" and a "neural network path submodule" set up in parallel.

[0033] First, within the rule path submodule, the system pre-loads and maintains a structured Traditional Chinese Medicine (TCM) knowledge graph, which contains prior diagnostic rules from authoritative TCM experts. During inference, the system extracts local soft labels of the tongue image (such as "pale tongue color: 0.91" and "teeth marks on the tongue: 0.88") output by the bimodal coding module, as well as the facial feature text extraction results. The rule path submodule uses these high-confidence local features as query entities, performing graph path traversal and association rule matching within the TCM knowledge graph. For example, when a rule node containing both "pale tongue color" and "teeth marks on the tongue" is matched, the system calculates the first constitution probability distribution based on the TCM expert rules using the confidence algorithm built into the graph. (For example, output a vector representing the probability of nine TCM constitutions, with the probability of Qi deficiency being significantly higher). This approach ensures the lower limit of the model's diagnosis, guaranteeing that the results do not deviate from traditional TCM principles.

[0034] Meanwhile, in the neural network path submodule, the system receives the 128-dimensional shared-space bimodal fusion features. To uncover implicit correlations in the data that cannot be fully explored manually, this submodule constructs a deep network incorporating a multilayer perceptron (MLP) and an attention mechanism. The fusion features are first transformed in dimension through multiple nonlinear fully connected layers, and then input to the attention mechanism layer. The attention mechanism layer adaptively assigns different importance weights to feature channels of different dimensions (for example, in determining a specific constitution, it automatically amplifies the weight of facial luster features and suppresses the weight of tongue coating thickness). After the aforementioned data-driven deep feature extraction, this submodule calculates the probability distribution of the second constitution. This approach endows the system with the ability to generalize to handle complex, nonlinear, and atypical cases.

[0035] After obtaining the two outputs, the key breakthrough of this invention lies in the scientific fusion of them. This embodiment abandons the conventional static weight allocation method (such as a fixed 0.5 to 0.5 weighting) and instead introduces a learnable dynamic gating mechanism. This mechanism can dynamically evaluate which side, the "rule" or the "neural network," is more reliable for each specific patient input.

[0036] In terms of specific calculation logic, the system first calculates the probability distribution of the first physical constitution. With the probability distribution of the second constitution Vector feature concatenation is performed to obtain a joint state vector. This joint state vector is then input into a gating network with one or more fully connected layers and the sigmoid function as the activation function to calculate the dynamic gating weight factor. Its core calculation formula is represented as follows: in, Let be the learnable weight matrix of this gated network. For bias terms, This represents the concatenation operation of probability vectors. This represents the Sigmoid nonlinear activation function.

[0037] Calculating the dynamic weighting factor Then, the system dynamically weights and sums the two outputs to obtain the final probability distribution output of the hybrid routing fusion module. Its calculation formula is expressed as: In this dynamic gating architecture, the weight matrix During the initial model training phase of the system, continuous updates and optimizations will be performed using backpropagation algorithm based on sample data labeled by real doctors. From a technical perspective, when the input features are extremely typical and perfectly match the rules of ancient Chinese medicine texts, the gating network tends to output larger values. The value makes the final result highly dependent on the expert's knowledge graph, ensuring the authority of the diagnosis; however, when the input features are fuzzy, belong to difficult and complicated cases, or are long-tail cases not covered by the knowledge graph, the gating network will output a smaller value. This value allows the neural network path, with its powerful generalization ability, to dominate the diagnosis. This dynamic flow mechanism not only significantly improves the overall accuracy of constitution identification (based on tests using a self-built dataset of 12,000 cases, the Top-1 accuracy can jump from the current 72.3% to 89.7%), but also fundamentally breaks through the technical bottleneck in the field of TCM AI where "accuracy" and "interpretability" are mutually exclusive.

[0038] Furthermore, the specific operation process of the generative conditioning engine is as follows: After the hybrid routing fusion module outputs the final constitution probability distribution result, the system flow enters the generative conditioning engine module. Existing TCM auxiliary systems typically use static rule matching to generate uniform conditioning suggestions, or directly call unconstrained large language models. This not only ignores individual patient differences and environmental changes, but also easily leads to serious medical safety risks due to the "illusion" of large models. To address the above technical problems, this invention proposes a closed-loop generation architecture that integrates dynamic prompt word construction, rigorous medical safety verification, and deep interpretability echo.

[0039] Specifically, the generative conditioning engine first assembles the context through a dynamic construction submodule using internal prompts. The system not only receives the final constitution identification result (e.g., "primarily Qi deficiency, with some damp-heat"), but also simultaneously extracts multi-dimensional status data of the current user through a front-end interface or user profile database. This data includes: real-time clinical symptom information (e.g., "fatigue, excessive sweating"), user physiological profile (e.g., "age 45, female"), current seasonal characteristics (e.g., "late summer, high humidity"), and geographical location information (e.g., "Lingnan region"). The system maintains a preset prompt template pool. Through dynamic placeholder replacement technology, it concatenates and semantically fuses the above multi-dimensional information to generate highly personalized, structured prompts.

[0040] Subsequently, the personalized suggestion is input into the large language model calling submodule. To ensure that the output can be directly parsed and utilized by downstream medical business systems, this embodiment imposes strict format constraints on the large language model at the API call level, mandating that it output treatment suggestions in a preset data interaction format. This structured output is limited to content blocks covering five dimensions: dietary dos and don'ts, daily life regulation, exercise methods (such as recommended Baduanjin), emotional regulation, and meridian acupoint massage guidance.

[0041] Addressing the core security concerns in medical applications, this embodiment incorporates a robust suggestion security verification layer at the LLM output. This verification layer includes a regular expression rule library and a medical sensitive word dictionary, manually reviewed and updated by a team of professional TCM doctors. After receiving the initial structured suggestions generated by the LLM, the verification layer performs a full scan of the text using a multi-pattern string matching algorithm. For example, if it identifies exaggerated or inappropriate terms such as "prescription," "guaranteed cure," or "absolutely effective," or recommends aggressive procedures like "blood-activating and stasis-removing" to users with a pregnant profile, the verification layer will directly block the suggestion and, through a feedback loop, require the LLM to regenerate it, or directly replace it with a system-preset safety fallback statement. This mechanism completely eliminates the uncontrollable risks of large language models in serious medical scenarios.

[0042] Furthermore, to completely address the "black box" pain point of AI diagnosis and establish a foundation of trust between doctors and patients, the generative conditioning engine innovatively introduces an interpretable echo mechanism based on Shapley and interpretation algorithms. While arriving at a constitution conclusion, the system retrospectively calculates the specific numerical contribution of each local input feature (such as tongue color and facial luster) extracted by the bimodal coding module to the final constitution probability output.

[0043] For the Each input feature contributes to the SHAP value of the final model output. The calculation formula is represented as: in, Represents the set of all input features. Indicates that it does not contain features Any feature subset, , representing the number of features in the set, Indicates using only a subset of features The predicted output value of the time model. This formula calculates the features. The weighted average of marginal contributions across all possible combinations of feature subsets enables precise quantification of the importance of features in nonlinear complex models.

[0044] After completing the above calculations, the engine maps these numerical contributions to a visual heatmap or structured explanatory text, and displays it back to the user's front-end interface. For example, in the diagnostic report of the front-end app, the system not only informs the user that they have a "Qi deficiency constitution," but also clearly marks it with highlighted blocks or text: "Because your tongue appearance shows 'pale white tongue color' (contribution to the determination of Qi deficiency +18%) and is accompanied by 'teeth marks on the tongue' (contribution +15%), this conclusion is reached." According to self-built clinical double-blind test data, after introducing this interpretability mechanism, the subjective satisfaction score of professional doctors and patients with the system's diagnostic results jumped significantly from 3.2 points in the traditional black-box model to 4.6 points (out of 5), achieving extremely significant technological progress and practical benefits.

[0045] Furthermore, the specific operation process of the online-offline closed-loop update module is as follows: In practical applications of medical artificial intelligence, traditional assisted diagnostic models often face the technical challenge of changing user group characteristics over time (i.e., "data drift" or "concept drift") after deployment, leading to a continuous decline in model accuracy over long-term operation. Furthermore, medical scenarios place extremely high demands on system stability and security; any unverified model update could potentially trigger serious medical accidents. To completely address the aforementioned technical pain points of "model aging" and "high update risk," this embodiment introduces an online-offline closed-loop update and continuous learning module at the system architecture's underlying level.

[0046] Specifically, when the system provides users with online constitution identification and conditioning generation services, its data drift detection submodule asynchronously and non-blockingly writes and stores the user-uploaded anonymized image data, extracted bimodal feature vectors, and final inference logs into the underlying data storage system (e.g., using a Delta Lake data lake architecture that supports ACID transactions) in the background. To enable the system to automatically detect whether its knowledge is outdated, this submodule periodically runs a rigorous mathematical distribution verification algorithm in the background to compare the distribution differences between the newly added online input data features and the historical training set data features.

[0047] This embodiment employs a dual detection algorithm combining the Kolmogorov-Smirnov test (KS-test) and feature embedding distance. First, the KS-test is used to evaluate the difference in the cumulative distribution function (CDF) of one-dimensional independent features. Let the empirical cumulative distribution function of a feature in the historical training set be... The cumulative distribution function of recent online data is Then the KS statistic The calculation formula is represented as: when Greater than the preset statistical threshold (e.g.) When the threshold value at the significance level is reached, the system initially determines that there is a feature level drift.

[0048] Meanwhile, in order to capture the complex semantic shifts in the high-dimensional feature space, the system further calculates the online data feature embedding center. Embedded center with training set features The cosine distance between them serves as a metric for high-dimensional data drift. Its calculation formula is expressed as: Only when the above low-dimensional Statistics and High Dimension When the indicators simultaneously exceed the system's preset tolerance threshold, or when the manual review backend has accumulated a sufficient number of newly added difficult case samples that have been labeled with high quality by professional TCM doctors, the system will automatically trigger a retraining warning instruction for the dual-modal coding module (such as the underlying ResNet feature extractor) and the hybrid routing fusion module, and start offline incremental training.

[0049] To prevent the deployment of substandard models due to overfitting or catastrophic forgetting, this embodiment incorporates a rigorous automated evaluation and interception submodule. After incremental training generates candidate models, the system must blindly test them on a reserved offline golden test set containing difficult samples. In addition to comparing conventional macro-accuracy metrics such as F1 scores, the system must also perform rigorous statistical significance tests.

[0050] Specifically, the system performs a paired t-test on the difference in prediction errors between the current online old model and the candidate new model on the test set. Only when the key performance indicators of the new model are superior to those of the old model in absolute value, and this improvement is statistically significant, will the new model be considered successful. Only when the significance test threshold is reached will the system mathematically and logically determine that the model iteration is truly effective, rather than an accidental improvement caused by the randomness of the test set sampling, and thus allow the new model to enter the release queue.

[0051] Finally, under the control of the canary release and configuration management submodule, the verified candidate models are tagged with versions through the model registry center. The release process adopts a smooth, low-volume A / B testing strategy: in the initial stage, only, for example, 5% of online requests are routed to the new model, and its latency, error rate, and user feedback in real business scenarios are monitored in real time; after all monitoring indicators stabilize, the release is gradually increased until a full and seamless switch is achieved. The entire update process is transparent to online services and requires no downtime or service interruption.

[0052] In addition to updating the weights of the deep learning model, this embodiment creatively introduces a version control mechanism based on code-based operations to address the prompt word template pool in the generative conditioning engine, which is highly susceptible to business fluctuations. All system-level prompt words for the large models are stored as declarative configuration files in a Git repository. When medical experts need to adjust the generation logic of conditioning suggestions (e.g., updating seasonal dietary restrictions for a specific constitution), they only need to submit a Pull Request (PR). If a new version of the Prompt causes output format errors in the large models or triggers frequent interception warnings from the security verification layer, the system can perform hot updates and rollbacks of the prompt word version within seconds based on Git records, thus providing a final safety net for the medical AI system.

[0053] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A system for intelligent identification of TCM constitution and generation of dynamically interpretable treatment plans, characterized in that: include: A bimodal coding module is used to extract the features of the tongue image and the facial image input by the user, and align the tongue image features and the facial image features to a shared embedding space to generate bimodal fusion features; The hybrid routing fusion module is used to receive the dual-modal fusion features and input them in parallel to the rule path based on the TCM knowledge graph and the neural network path based on deep learning. The output probabilities of the rule path and the neural network path are dynamically weighted and fused through a learnable gating mechanism to obtain the final constitution identification result. A generative conditioning engine is used to construct prompt words based on the final constitution identification results and preset individual information, and to call a large language model to generate structured conditioning suggestions. Simultaneously, it outputs interpretable information reflecting the contribution of each input feature to the physical fitness identification result; The online-offline closed-loop update module is used to collect image data and inference logs from online detection, trigger model retraining based on data drift detection results or newly added samples that have been manually reviewed, and automatically update and deploy the new model to the online system after it passes offline evaluation.

2. The TCM constitution intelligent identification and dynamically interpretable conditioning plan generation system according to claim 1, characterized in that, The bimodal coding module includes a tongue image coding subnet; The tongue image encoding subnetwork adopts a hybrid network architecture of convolutional neural network and Transformer; wherein, the convolutional neural network is used to extract local visual features of the tongue image, and the Transformer is used to extract global dependency features of the tongue image based on a multi-head attention mechanism; the hybrid network outputs multi-dimensional soft label features, which represent the probability distribution of local attributes of the tongue image, specifically including the probability values ​​of tongue color, tongue shape, coating color and coating texture.

3. The TCM constitution intelligent identification and dynamically interpretable conditioning plan generation system according to claim 2, characterized in that, The bimodal coding module also includes a face coding subnet; The facial encoding subnet is constructed based on a visual large language model. The facial encoding subnet receives facial images and extracts prompt words by combining preset facial features, and outputs descriptive text containing facial complexion and luster attributes. Then, the descriptive text is parsed and transformed into facial structured features of preset dimensions through a preset mapping layer.

4. The TCM constitution intelligent identification and dynamically interpretable conditioning scheme generation system according to claim 3, characterized in that, The bimodal coding module also includes a feature alignment layer; The feature alignment layer is used to concatenate the soft label features output by the tongue image encoding subnet with the facial structured features output by the face encoding subnet, and map them to a shared embedding space of the same dimension. During the model training phase of the feature alignment layer, the cosine contrast loss function is used to constrain and optimize the model in order to narrow the semantic distance between the tongue image features and facial features of the same user in the shared embedding space and reduce cross-modal differences. The system also includes an input preprocessing module; the input preprocessing module is configured to receive tongue and facial images of any size uploaded by the user, and then uniformly scale them to a preset standard resolution size before distributing them to the bimodal encoding module.

5. The TCM constitution intelligent identification and dynamically interpretable conditioning plan generation system according to claim 4, characterized in that, The hybrid routing fusion module includes a rule-based path submodule; The rule path submodule contains a pre-built TCM knowledge graph. The rule path submodule is configured to: extract local attribute labels from the dual-modal fusion features, perform association queries and matching in the traditional Chinese medicine knowledge graph, and calculate the first constitution probability distribution based on traditional Chinese medicine expert rules. .

6. The TCM constitution intelligent identification and dynamically interpretable conditioning scheme generation system according to claim 5, characterized in that, The hybrid routing fusion module further includes a neural network path submodule; the neural network path submodule includes a multilayer perceptron and an attention mechanism network layer; the neural network path submodule is configured to: receive the dual-modal fusion features, perform nonlinear feature transformation through the multilayer perceptron, and dynamically assign importance to different feature dimensions using the attention mechanism to calculate a data-driven second physical probability distribution. ; The specific configuration of the learnable gating mechanism is as follows: The first physical probability distribution The second physical probability distribution Perform vector concatenation; The dynamic gating weight factor is calculated using a single-layer or multi-layer fully connected network with an activation function. Its calculation formula is expressed as: in This represents the learnable weight matrix of the network; Using the dynamic gating weight factor The outputs of the rule path and the neural network path are dynamically weighted and summed to obtain the final physical probability distribution, which is represented by the following formula: Wherein, the learnable weight matrix During the model training phase of the system, continuous updates and optimizations are performed using the backpropagation algorithm.

7. The TCM constitution intelligent identification and dynamically interpretable conditioning plan generation system according to claim 6, characterized in that, The generative conditioning engine includes a prompt word dynamic construction submodule and a large language model invocation submodule; The prompt word dynamic construction submodule maintains a preset prompt word template pool; the prompt word dynamic construction submodule is configured to: obtain the final constitution identification result, and simultaneously extract the current user's symptom information, user profile, current seasonal characteristics and geographical location information; fill the above multi-dimensional information into the corresponding prompt word template, and combine to generate structured individualized prompt words; The large language model calling submodule inputs the individualized prompt words into the pre-deployed large language model and constrains the large language model to output structured conditioning suggestions containing multiple dimensions in a preset data interaction format.

8. The TCM constitution intelligent identification and dynamically interpretable conditioning plan generation system according to claim 7, characterized in that, The generative conditioning engine also includes a suggested security verification layer; The suggested security verification layer is located at the output end of the large language model calling submodule; the suggested security verification layer is internally configured with a regular expression rule library and a medical sensitive word library; The security verification layer is configured to: before outputting the structured conditioning suggestions to the user's front end, use the regular expression rule library and the medical sensitive word library to match and filter the text content of the structured conditioning suggestions in order to intercept or correct illegal suggestions that pose medical safety risks; Explainable information is obtained and displayed in the following ways: The Shapley algorithm is used to calculate the specific numerical contribution of each local input feature extracted by the dual-modality coding module to the final physical fitness identification result output by the hybrid routing fusion module. The numerical contribution is displayed back to the user's front-end interface in the form of a visual heatmap or structured text to show the user the driving weight of specific tongue or facial features in diagnosing a specific constitution.

9. The TCM constitution intelligent identification and dynamically interpretable conditioning scheme generation system according to claim 8, characterized in that, The online-offline closed-loop update module includes a data drift detection submodule; The data drift detection submodule is configured to: synchronize the image data features and inference result logs of the user's online detection input to the underlying data storage system in real time or periodically; and calculate the distribution difference between the newly added online input data features and the historical training set data features. The data drift detection is calculated using an algorithm that combines the Kolmogorov-Smirnov test with feature embedding distance. When the detected distribution difference exceeds a preset drift threshold, a retraining warning instruction is automatically triggered for the dual-modal coding module and the hybrid routing fusion module.

10. The TCM constitution intelligent identification and dynamically interpretable conditioning scheme generation system according to claim 9, characterized in that, The online-offline closed-loop update module also includes an automated evaluation and interception submodule; The automated evaluation and interception submodule is configured to: upon receiving a retraining warning instruction or after manually reviewing and annotating new high-quality training samples, initiate incremental training or retraining of the model to generate candidate version models. The candidate version model is evaluated against the current online model on an offline reserved test set based on key performance indicators. Only when the key performance indicators of the candidate version model are superior to those of the current online model and pass the preset statistical significance test will the candidate version model be considered successful. When the candidate version model is deemed valid, it is allowed to proceed to the next step of the automatic update process. It also includes a canary release and configuration management submodule; the canary release and configuration management submodule manages model versions based on the model registry center; after the candidate version model is determined to be valid, it is automatically deployed to the canary environment for small-scale A / B testing; and after the test indicators are stable, a seamless traffic switch for full online service is performed. For the prompt word template pool in the generative conditioning engine, a code-based operation and maintenance mechanism is used for version control and update management to support rapid rollback of prompt word versions.

Images