Intelligent diagnosis and treatment system for rheumatoid arthritis and its comorbidities in traditional Chinese and western medicine and computer equipment

Abstract

The present application belongs to the technical field of new medical instruments crossing medical artificial intelligence and intelligent devices, and relates to a kind of rheumatoid arthritis and its comorbidity traditional Chinese and western medicine intelligent diagnosis and treatment system and computer equipment;System includes:Perception layer, for collecting the multi-modal data of the object to be diagnosed and treated;Decision layer, for receiving the multi-modal data collected by the perception layer, and calling the pre-constructed medical special multi-modal large model, respectively executing the combined diagnosis of traditional Chinese and western medicine for rheumatoid arthritis and its comorbidity, to output the corresponding diagnosis result;Treatment module, for receiving the traditional Chinese medical syndrome diagnosis result and / or western medicine diagnosis result output by the decision layer, dynamically selecting the treatment mode matched with the object to be diagnosed and treated, to provide the corresponding rehabilitation treatment scheme.The present application realizes the closed loop of standardized and intelligent diagnosis and treatment operation, breaks through the limitation of traditional manual operation or semi-automatic execution, and improves the overall diagnosis and treatment efficiency and quality.

Description

Technical Field

[0001] This invention relates to the field of novel medical device technology that combines medical artificial intelligence and intelligent devices, and in particular to an intelligent diagnosis and treatment system and computer equipment for rheumatoid arthritis and its comorbidities using both traditional Chinese and Western medicine. Background Technology

[0002] In current technology, Western medicine mainly relies on imaging and laboratory tests to assess the condition, while traditional Chinese medicine is based on diagnosis and treatment based on methods such as observation, auscultation, inquiry, and palpation.

[0003] However, existing traditional clinical diagnosis and treatment models have many shortcomings, specifically in the following aspects: First, traditional diagnosis and treatment suffers from significant variations in manual operation and insufficient precision; procedures such as ultrasound scanning and pulse diagnosis heavily rely on the experience and proficiency of medical staff, easily leading to inconsistencies in operating force, angle, and trajectory, resulting in deviations in diagnostic and treatment results. Second, traditional diagnosis and treatment operations are inefficient and lack standardized procedures, requiring medical staff to manually adjust equipment and control the operating rhythm, which is not only time-consuming but also prone to incomplete data collection due to non-standard operation. Third, traditional treatment interventions lack personalization and have poor safety; physical therapy parameters are mostly fixed settings, making it difficult to adapt to the differences in different patients' conditions and unable to respond promptly to changes in the condition, easily leading to poor treatment effects or worsening of the condition; at the same time, drug treatment is prone to problems such as missed medication and dosage deviations. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide an intelligent diagnosis and treatment system and computer equipment for rheumatoid arthritis and its comorbidities using both traditional Chinese and Western medicine, in order to solve the problems of large differences in manual operation, low operation efficiency, non-standard procedures, and insufficient personalization of traditional treatment interventions in the existing technology.

[0005] This invention provides an intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities, comprising: The perception layer is used to call a pre-built medical-specific multimodal large model to control the multimodal acquisition module to collect multimodal data of the object to be diagnosed and treated. The multimodal data includes at least: synovial hypertrophy data, blood flow characteristic data, bone microstructure data, soft tissue characteristic data, traditional Chinese medicine signs data, and physiological and pathological data. The decision layer receives multimodal data collected by the perception layer and invokes the medical-specific multimodal large model to perform integrated traditional Chinese and Western medicine diagnostic operations for rheumatoid arthritis and its comorbidities, outputting corresponding traditional Chinese medicine syndrome diagnosis results and / or Western medicine diagnosis results; and The treatment module is used to receive the diagnostic results output by the decision layer, dynamically select the treatment mode that matches the patient, and provide a corresponding rehabilitation treatment plan.

[0006] Furthermore, the multimodal acquisition module includes at least: The first feature data acquisition module is used to acquire the synovial thickening data and the blood flow feature data through a first multimodal sensor group configured by a preset dexterous hand; The second feature data acquisition module is used to acquire the TCM vital signs data through the second multimodal sensor group configured by the dexterous hand; The feature data acquisition module is used to connect to external medical imaging equipment and acquire the bone microstructure data and the soft tissue feature data; it is also used to acquire the physiological and pathological data through the interface of peripheral detection equipment.

[0007] Furthermore, the TCM physical examination data includes at least: tongue appearance characteristics and facial complexion characteristics, and pulse characteristics; The physiological and pathological data include at least: body temperature, RDW-SD standard deviation of red blood cell distribution width, CREA creatinine, CRP index, ESR erythrocyte sedimentation rate, WBC white blood cell count, PLT platelet count, IL-6 index, IL-8 index, LD lactate dehydrogenase, and LYM lymphocyte count.

[0008] Furthermore, the training tasks of the medical-specific multimodal large model in the perception phase include: Based on expert detection operation samples and annotations of patients with rheumatoid arthritis and their comorbid history, the multimodal data obtained by controlling the multimodal acquisition module to perform the multimodal data acquisition operation is used as the training target, and a supervised learning algorithm is used to iteratively train the medical-specific multimodal large model.

[0009] Furthermore, the training tasks of the medical-specific multimodal large model during the decision-making phase include: Based on historical data of synovial thickening, blood flow characteristics, bone microstructure, and soft tissue characteristics, and using the subclinical inflammation degree and microerosion volume of rheumatoid arthritis and its comorbidities as output labels, an LSTM long short-term memory network is used to construct and train a first structure-activity relationship model, so that the medical-specific multimodal large model has the ability to diagnose subclinical inflammation and microerosion volume. Based on historical data of TCM physical signs, and using traditional TCM syndromes as output labels, a second structure-activity relationship model is constructed and trained using a supervised learning algorithm, so that the medical-specific multimodal large model has the ability to diagnose traditional syndromes. Based on historical data of TCM physical signs and physiological and pathological data, cold syndrome and heat syndrome are used as group labels. PCA principal component analysis and OPLS-DA orthogonal partial least squares discriminant analysis algorithms are used to construct and train a group classification model so that the medical-specific multimodal large model has the ability to diagnose cold and heat syndromes.

[0010] Furthermore, the decision-making layer includes a Western medicine diagnostic module and a Traditional Chinese Medicine (TCM) diagnostic module. These modules can respectively invoke a medical-specific multimodal large-scale model that has completed decision-making training to complete the corresponding diagnosis. The Western medicine diagnostic module is used to output Western medicine diagnostic results of subclinical inflammation and micro-erosion volume based on the synovial thickening data, blood flow characteristic data, bone microstructure data and soft tissue characteristic data. The TCM diagnostic module is used to output TCM syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the TCM physical signs data and the physiological and pathological data.

[0011] Furthermore, the TCM diagnostic module specifically includes: a traditional syndrome diagnosis module and a cold / heat syndrome diagnosis module; The traditional syndrome diagnosis module is used to output traditional syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the tongue appearance feature and facial color feature data and the pulse feature data. The cold and heat syndrome diagnosis module is used to output the cold and heat syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the TCM physical signs data and the physiological and pathological data.

[0012] Furthermore, the decision-making layer also includes: a physiological characteristic scoring module; The physiological feature scoring module is used to perform grayscale scoring on the synovial thickening data, color Doppler scoring on the blood flow feature data, and transmit the corresponding scoring results to the medical-specific multimodal large model for auxiliary diagnosis.

[0013] Furthermore, the treatment module includes at least: a drug treatment mode and a physical therapy mode; The drug treatment mode is used to assist in screening drugs from the Chinese medicine database and the Western medicine database based on the diagnosis results of the TCM syndrome and / or the diagnosis results of the Western medicine, and output a personalized drug treatment plan after receiving the doctor's review and confirmation. The physical therapy mode is used to generate personalized massage plans based on the Western medicine diagnosis results of the subclinical inflammation and micro-erosion volume, and to control the dexterous hand to perform traditional Chinese medicine massage therapy.

[0014] Furthermore, the execution process of the physical therapy mode specifically includes: Based on the Western medicine diagnosis results of the subclinical inflammation and micro-erosion volume, and combined with the soft tissue characteristic data, the cartilage damage of the damaged site of the subject to be diagnosed and treated is graded. Based on the location of the injury and the corresponding cartilage injury grading results, the corresponding massage intensity, frequency, duration and technique are retrieved from the preset massage program library to generate the personalized massage program. The personalized massage plan is converted into control commands to control the dexterous hand to perform massage operations on the injured area, and pressure data is fed back in real time to dynamically adjust the massage intensity. After each massage, multimodal data is recollected through the perception layer, and the massage effect is evaluated by the medical-specific multimodal large model. Based on the results of multiple massage evaluations, the operation parameters are iteratively optimized to form a new personalized massage plan and execute subsequent massage operations.

[0015] This invention also provides a computer device, including at least one processor and at least one memory communicatively connected to the processor; The memory stores instructions that can be executed by the processor to implement the intelligent diagnosis and treatment system for rheumatoid arthritis and its comorbidities using both traditional Chinese and Western medicine, as described in any of the preceding claims.

[0016] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: First, unlike related technologies where there are significant differences in manual operation, this invention achieves accurate acquisition of multimodal and multidimensional data through a sensing layer, improving the standardization of acquisition processes such as synovial thickening and blood flow characteristics, and reducing acquisition errors. At the same time, it accurately acquires physiological characteristics such as tongue appearance, facial color, and pulse, avoiding the subjectivity and operational deviations of manual acquisition, ensuring the standardization, integrity, and reliability of the acquired data, and providing high-quality data support for subsequent diagnosis.

[0017] Secondly, unlike related technologies which suffer from insufficient diagnostic accuracy, this invention, through a decision-making layer, achieves precise scoring of physiological characteristics, including grayscale synovial thickening and color Doppler blood flow, with high accuracy and consistent results. By deeply integrating Western medical imaging diagnosis with traditional Chinese medicine syndrome differentiation and treatment, it can accurately diagnose subclinical inflammation and bone microerosion volume, clarify the degree of lesions, and simultaneously achieve the objective and standardized determination of traditional Chinese medicine syndromes such as "qi deficiency and blood stasis" and "yang deficiency and cold coagulation," as well as cold and heat syndromes, thus avoiding the subjectivity of traditional Chinese medicine syndrome differentiation and improving the accuracy and repeatability of syndrome diagnosis.

[0018] Third, unlike related technologies which suffer from insufficient personalization and compromised safety, this invention achieves intelligent output of personalized treatment plans through a combination of drug and physical therapies. Drug therapy assists in screening suitable medications and provides drug reference information, helping doctors improve the safety and targeting of medication use. Physical therapy dynamically adjusts the massage plan according to the degree of cartilage damage, avoiding secondary injury. This forms a closed-loop treatment process of data collection, diagnosis, and treatment. While improving effectiveness and personalization, it overcomes the limitations of traditional manual operation or semi-automated execution, utilizing intelligent equipment for complete automation without human intervention. This provides a new model for assisted medicine, achieving a fully automated closed loop from data collection and analysis to intervention plan generation. The output results are used to assist clinical medical care, and the final treatment is confirmed and executed by the physician. This enhances the system's information processing capabilities, thereby improving overall treatment efficiency and quality. In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0019] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 is a schematic diagram of the intelligent diagnosis and treatment system for rheumatoid arthritis and its comorbidities using both traditional Chinese and Western medicine, according to an embodiment of the present invention. Detailed Implementation

[0020] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0021] One specific embodiment of the present invention discloses an intelligent integrated traditional Chinese and Western medicine diagnostic and treatment system for rheumatoid arthritis and its comorbidities. Rheumatoid arthritis is an autoimmune disease characterized primarily by chronic, symmetrical polyarticular synovitis, which can lead to progressive destruction of articular cartilage and bone tissue. During the disease progression, patients often experience one or more other diseases related to the pathogenesis, inflammatory state, or treatment of rheumatoid arthritis; these are called comorbidities, such as cardiovascular disease, osteoporosis, interstitial lung disease, Sjögren's syndrome, and metabolic syndrome. Figure 1 As shown, this system specifically includes the following modules: The perception layer is used to call a pre-built medical-specific multimodal large model to control the multimodal acquisition module to collect multimodal data of the object to be diagnosed and treated. The multimodal data includes at least: synovial hypertrophy data, blood flow characteristic data, bone microstructure data, soft tissue characteristic data, traditional Chinese medicine signs data, and physiological and pathological data. The decision layer receives multimodal data collected by the perception layer and invokes the medical-specific multimodal large model to perform integrated traditional Chinese and Western medicine diagnostic operations for rheumatoid arthritis and its comorbidities, outputting corresponding traditional Chinese medicine syndrome diagnosis results and / or Western medicine diagnosis results; and The treatment module is used to receive the diagnostic results output by the decision layer, dynamically select the treatment mode that matches the patient, and provide a corresponding rehabilitation treatment plan.

[0022] Specifically, the perception layer, decision layer, and treatment module communicate with each other. In practical applications, the perception layer first collects multimodal data of the patient based on the user-inputted target command for the current data acquisition task (e.g., commands to perform data acquisition or intelligent diagnosis of rheumatoid arthritis and its comorbidities). After receiving the multimodal data, the decision layer uses an internally configured medical-specific multimodal model to perform Western medicine and Traditional Chinese Medicine diagnostic operations for rheumatoid arthritis and its comorbidities, outputting corresponding diagnostic results. When rheumatoid arthritis (and its comorbidities) is diagnosed, the treatment module provides a treatment mode matching the patient based on the diagnostic results and determines a corresponding rehabilitation treatment plan, thereby achieving intelligent diagnosis and treatment combining Western and Traditional Chinese Medicine.

[0023] It should be noted that, since the decision layer of this invention can output multiple diagnostic results, including Western medicine diagnostic results (subclinical inflammation degree and micro-erosion volume), traditional syndrome diagnostic results (such as qi deficiency and blood stasis, yang deficiency and cold coagulation, etc.), and cold and heat syndrome diagnostic results (cold syndrome or heat syndrome), based on this, for the diagnosis and disease assessment of rheumatoid arthritis (and its comorbidities), this system supports multiple combined judgment modes, specifically including: If only the extent of joint structural damage is needed, Western medical diagnostic results can be used alone. If guidance for traditional Chinese medicine treatment is required, traditional Chinese medicine syndrome diagnosis or cold / heat syndrome diagnosis can be used alone. Alternatively, both types of Chinese medicine diagnostic results can be used simultaneously to achieve mutual verification between syndromes and the Eight Principles of Traditional Chinese Medicine. Of course, Western and Chinese medicine diagnostic results can also be used in combination without restriction. For example, when Western medicine diagnoses subclinical inflammation and Chinese medicine diagnoses damp-heat stagnation syndrome, this system can comprehensively determine that the disease is in an active phase and recommend a combined Chinese and Western medicine treatment plan. In addition, this system outputs confidence scores for all types of diagnostic results. When the confidence score of a single diagnosis is low, it can automatically trigger cross-validation of diagnostic results from other dimensions or prompt for supplementary data collection.

[0024] Furthermore, in the treatment module, the system automatically triggers different treatment modes based on the TCM / Western medicine diagnostic results output by the decision layer. The determination of the treatment mode can be achieved by the mapping function obtained by the medical-specific multimodal large model through supervised learning training during the treatment phase, or it can be determined according to a preset correspondence. Specifically, when the TCM diagnostic results output by the decision layer show "Qi deficiency and blood stasis" or "Yang deficiency and cold coagulation," or "cold syndrome" or "heat syndrome," the system will prioritize the drug treatment mode. When the Western medicine diagnostic results output by the decision layer show the presence of subclinical inflammation or micro-erosion volume, the system will prioritize the physical therapy mode.

[0025] It is worth noting that the treatment modes and diagnostic results in this system are not in a fixed one-to-one correspondence. Although under normal circumstances, Western medicine diagnoses (such as inflammation or micro-erosion) mainly trigger physical therapy or Western medicine treatment, and traditional Chinese medicine diagnoses mainly guide the selection of Chinese herbal medicines, the system supports dynamic matching strategies. For example, when a traditional Chinese medicine diagnosis indicates "qi stagnation and blood stasis" and there is significant local pain, a physical therapy mode can be activated to perform acupressure to unblock the meridians; conversely, when a Western medicine diagnosis only shows mild cartilage damage without significant inflammation, a drug mode such as external application of Chinese herbal medicine can be used for intervention. In addition, based on the patient's wishes, the severity of the condition, and feedback on treatment effects, a medical-specific multimodal large model or professional physicians can flexibly select the combination of treatment modes corresponding to the traditional Chinese and Western medicine diagnoses, such as selecting a single mode (drug therapy only or physical therapy only) or a combined mode (drug therapy and physical therapy simultaneously), to achieve individualized dynamic treatment.

[0026] In some implementations, the perception layer, decision-making layer, and treatment module all rely on and are controlled by a medical-specific multimodal large model. The medical-specific multimodal large model can be based on an architecture that integrates the ViT-Unet model (visual Transformer-U-shaped network) and the medical LLM large language model, and is trained using supervised learning. The ViT-Unet model is responsible for handling visual tasks, such as segmentation of joint ultrasound image data, CT / MRI image data, and tongue image data; the LLM large language model is responsible for handling language tasks, such as logical reasoning in traditional Chinese medicine diagnosis and treatment plan generation.

[0027] Furthermore, during the training process, the large model is divided into multiple training stages, each corresponding to different training objectives and task configurations; among them, the training tasks of the medical-specific multimodal large model in the perception stage include: Based on expert detection operation samples and annotations of patients with rheumatoid arthritis and their comorbid history, the multimodal data obtained by controlling the multimodal acquisition module to perform the multimodal data acquisition operation is used as the training target, and a supervised learning algorithm is used to iteratively train the medical-specific multimodal large model.

[0028] For example, firstly, collect physician operation samples: Select 30-50 TCM physicians with more than 5 years of experience in the diagnosis and treatment of rheumatoid arthritis (and its comorbidities). Use high-definition camera equipment and sensor data recorders to simultaneously collect the detection operations of professional physicians on 100-150 patients with different degrees of synovial lesions. The recorded content includes: collection techniques, such as the force range of light pressure, sliding, and fixed-point pressing 1-5N; collection angle, such as the angle with the joint plane 30-60 degrees, with each 5-degree increment; collection process, such as first locating the joint pain point, then locating the suspected area of ​​synovial thickening, and finally locating the key area of ​​blood flow signal, with the collection time for each area being 3-5 seconds, etc. Secondly, large-scale model training and optimization are performed: the collected expert detection operation samples are labeled (such as labeling the type of technique, angle parameters, and process nodes) and input into a medical-specific multimodal large-scale model; that is, the training input in the perception stage is the expert detection operation samples (including collection techniques, collection angles, collection processes, etc.) of historical treatment subjects of rheumatoid arthritis (and its comorbidities), and the output is the standardized collection control strategy of the multimodal collection module (used to control the first and second feature data collection modules and feature data acquisition modules to execute the corresponding modal data collection). At the same time, a supervised learning algorithm is adopted, with "operational standardization" and "data collection accuracy" as evaluation indicators, and iterative training is conducted for 5000-8000 rounds. During the training process, 10%-15% of abnormal operation samples (such as angle deviation and technique errors) are added for anti-interference training to ensure that the model can accurately learn the standardized collection methods of physicians, so that the accuracy of the validation set is not less than 95% after training.

[0029] The training tasks of the medical-specific multimodal large model during the decision-making phase include: Based on historical data of synovial thickening, blood flow characteristics, bone microstructure, and soft tissue characteristics, and using the subclinical inflammation level and microerosion volume of rheumatoid arthritis and its comorbidities as output labels, an LSTM long short-term memory network is used to construct and train a first structure-activity relationship model, so that the medical-specific multimodal large model has the ability to diagnose subclinical inflammation and microerosion volume. Among them, subclinical inflammation refers to the synovial inflammation state of the joint that has not yet shown obvious clinical symptoms and can only be detected by imaging or molecular markers; microerosion volume is a volumetric indicator of the degree of bone microstructure damage quantitatively assessed by high-resolution imaging.

[0030] For example, a CNN-LSTM hybrid deep learning model, which integrates convolutional neural networks (CNN) and long short-term memory networks (LSTM), can be used to construct the first structure-activity relationship. Five-fold cross-validation is employed during model training to ensure the model's generalization ability. Preferably, 200 clinically confirmed cases (including subclinical inflammation and micro-erosions of varying degrees) can be used to validate the first structure-activity relationship model, ensuring that the accuracy of inflammation degree diagnosis is not less than 90% and the error in micro-erosion volume measurement does not exceed 10%.

[0031] Furthermore, based on historical data of TCM physical signs, and using traditional TCM syndromes as output labels, a second structure-activity relationship model is constructed and trained using a supervised learning algorithm, so that the medical-specific multimodal large model has the ability to diagnose traditional syndromes. Among them, traditional syndromes refer to the pathological generalizations with intrinsic correlations derived from the comprehensive judgment of physical signs such as tongue appearance, complexion, and pulse in TCM syndrome differentiation and treatment, such as "qi deficiency and blood stasis" and "yang deficiency and cold coagulation".

[0032] For example, in the syndrome feature mapping stage, 1,000-1,500 historical cases of rheumatoid arthritis (and its comorbidities) diagnosed by traditional Chinese medicine as "qi deficiency and blood stasis" or "yang deficiency and cold coagulation" are collected. Characteristic parameters such as tongue appearance (tongue color, tongue coating), complexion (redness, luster), and pulse (frequency, amplitude, rhythm) of each historical patient are extracted to establish a syndrome-feature parameter mapping library and clarify the threshold range of characteristic parameters corresponding to each syndrome.

[0033] Subsequently, in the second structure-activity relationship training phase, data from the mapping library is input into a medical-specific multimodal large-scale model. A supervised learning algorithm is used to construct a structure-activity relationship model between tongue appearance, facial color, pulse characteristics, and various traditional syndromes. During training, syndrome confusion samples (such as cross-cases of qi deficiency and blood stasis versus yang deficiency and cold coagulation) are added to improve the model's discrimination ability. Preferably, newly added clinical case data can be collected periodically to update the aforementioned syndrome and feature parameter mapping library, and the second structure-activity model can be iteratively trained to ensure that the diagnostic accuracy of each syndrome is not less than 88%.

[0034] Based on historical data of TCM physical signs and physiological and pathological data, cold syndrome and heat syndrome are used as group labels. PCA principal component analysis and OPLS-DA orthogonal partial least squares discriminant analysis algorithms are used to construct and train a group classification model so that the medical-specific multimodal large model has the ability to diagnose cold and heat syndromes. Among them, cold and heat syndromes are the basic classifications in the eight principles of TCM syndrome differentiation. Cold syndrome is manifested as aversion to cold and cold limbs, pale tongue and white coating, etc., caused by insufficient Yang Qi or excessive internal cold. Heat syndrome is manifested as fever, thirst, red tongue and yellow coating, etc., caused by excessive Yang heat or internal disturbance of heat.

[0035] For example, firstly, the historical data of various TCM physical signs and physiological and pathological data are standardized; then, the processed historical data are arranged in a preset order (such as the order of physiological indicators, tongue surface features, and pulse features) to form a high-dimensional matrix, with the matrix dimension being n × the total number of categories of historical data (n being the number of historical patient samples).

[0036] Secondly, the PCA algorithm is used to reduce the dimensionality of the high-dimensional matrix. The principal component contribution rate threshold is set to ≥85%, and the first 2-3 principal components are extracted to present the overall distribution trend of the data. Through the principal component analysis algorithm, noise in the data (such as abnormal physiological indicators, abnormal data caused by collection errors, etc.) is removed, core feature information is retained, and PCA score map is output to intuitively display the sample distribution.

[0037] Finally, using "cold syndrome" and "heat syndrome" as group labels (denoted as Y variable), and the core feature information processed by PCA as input (denoted as X variable), the OPLS-DA algorithm is used for group classification. Preferably, the number of cross-validations can be set to 10, and the algorithm parameters (such as the number of principal components and regularization coefficient) can be optimized to ensure that the group classification model has R²Y≥0.8 and Q²≥0.7, thus ensuring that the model has good interpretability and predictive ability. Here, R²Y represents the explanatory power of the model for the Y variable (i.e., group label, such as cold syndrome / heat syndrome), and Q² represents the predictive ability of the model.

[0038] It can be understood that the structure-activity relationship model in this embodiment refers to a mathematical model that establishes a mapping relationship between input feature data and output clinical status. PCA (Principal Component Analysis) is an unsupervised dimensionality reduction algorithm that transforms multiple original variables into a set of a few uncorrelated principal components through linear transformation, thereby preserving the main variation information of the data and removing noise. OPLS-DA (Orthogonal Partial Least Squares Discriminant Analysis) is a supervised classification algorithm that introduces orthogonal signal correction on the basis of partial least squares regression. It can maximize the difference between two classes (such as cold syndrome and heat syndrome) while eliminating variant samples irrelevant to classification, thus achieving efficient discrimination and grouping of samples. Further details are omitted here.

[0039] In this invention, a medical-specific multimodal large model learns the correlation between imaging features such as synovial thickening, blood flow signals, and bone microstructure and the degree of subclinical inflammation and micro-erosion volume, as well as the correlation between TCM signs and physiological and pathological data such as tongue appearance, complexion, and pulse and traditional syndromes and cold and heat syndromes, thereby realizing intelligent reasoning from multimodal data to TCM and Western medicine diagnostic results.

[0040] Furthermore, during the treatment phase, the medical-specific multimodal large model also requires targeted training to enable it to generate individualized treatment plans based on the diagnostic results output by the decision-making layer. These plans may include drug therapy, massage therapy, or a combination of both. Specifically, a large amount of historical diagnostic data on rheumatoid arthritis (and its comorbidities) patients and their corresponding effective treatment cases are collected. The Western medicine diagnostic results (such as subclinical inflammation level and micro-erosion volume) and traditional Chinese medicine diagnostic results (such as traditional syndrome types and cold / heat syndromes) of historical patients are used as input features. The physician-confirmed drug therapy plans (including the types, dosages, and usage of Chinese and Western medicines) and physical therapy plans (including massage acupoints, intensity, frequency, and duration) are used as output labels. Supervised learning algorithms are used to iteratively train the model. Positive and negative samples of drug interactions and massage effect feedback data are incorporated during training, enabling the model to learn to select personalized treatment modes under different syndrome and pathological states while ensuring medication safety.

[0041] After training, this multimodal large model can generate rehabilitation treatment plans combining traditional Chinese and Western medicine in practical applications and achieve dynamic optimization of treatment parameters. Of course, the specific implementation of the medical-specific multimodal large model can be adjusted by those skilled in the art and implemented with reference to architectures such as multi-label classification or joint learning, which will not be elaborated here.

[0042] In some preferred embodiments, the sensing layer of the present invention works in conjunction with the dexterous hand: the dexterous hand is used to simultaneously acquire joint local and whole-body characteristic data according to the instructions of the medical-specific multimodal large model.

[0043] It is understood that the dexterous hand is a robotic hand with a high degree of freedom and can be configured with a variety of sensors. Its fingertips are equipped with small musculoskeletal ultrasound probes, external flexible electronic skin sensors (to collect information on the contact between the probe and the skin) and six-dimensional force sensors (to collect information on the force and posture of the probe), a depth vision sensor is integrated in the wrist, and a pressure sensing module is integrated in the fingertips.

[0044] In some preferred embodiments, the multimodal acquisition module includes at least: a first feature data acquisition module, a second feature data acquisition module, and a feature data acquisition module. Further, the multimodal data processing unit built into the perception layer performs noise reduction, redundancy removal, and standardization processing on the aforementioned multimodal data. Using timestamp synchronization technology, it accurately aligns the image, vital signs, physiological, and pathological data, packages them into a standardized, structured dataset, and distributes it to the decision-making layer.

[0045] In the perception layer: the first feature data acquisition module is used to acquire the synovial hypertrophy data and the blood flow feature data through a first multimodal sensor group (small musculoskeletal ultrasound probe module) configured by a preset dexterous hand.

[0046] For example, a medical-specific multimodal large model can learn the acquisition methods of traditional Chinese medicine physicians using detection sensors, including acquisition techniques, acquisition angles, and acquisition procedures. This enables the first feature data acquisition module to dynamically acquire ultrasound grayscale (GS) synovial hypertrophy images and color Doppler (CD) blood flow signals when scanning the patient's metacarpophalangeal joints (MCP) and proximal interphalangeal joints (PIP), and simultaneously record parameters such as probe operation force and angle.

[0047] Furthermore, the second feature data acquisition module is used to acquire the TCM vital sign data through the second multimodal sensor group configured by the dexterous hand. The second multimodal sensor group specifically includes: a depth vision sensor, a pressure sensing module, etc. The TCM vital sign data includes at least: tongue appearance feature data, facial color feature data, and pulse feature data. That is, tongue appearance feature data and facial color feature data can be acquired through the depth vision sensor, and pulse feature data can be acquired through the pressure sensing module.

[0048] For example, the depth vision sensor is activated to capture high-definition images of the patient's face and tongue, accurately capturing detailed features such as tongue color, coating thickness, cracks, and facial texture. The dexterous hand switches to the pressure sensing module, simulating the traditional Chinese medicine technique of "lifting, pressing, and searching," applying different pressures to the cun, guan, and chi positions of the patient, dynamically collecting pulse waveform data and automatically extracting quantitative parameters such as enhancement index and maximum pressure change rate. Furthermore, the pressure sensor is also used to collect feedback pressure parameters when the dexterous hand performs massage operations.

[0049] Furthermore, the feature data acquisition module is used to connect to external medical imaging equipment and acquire the bone microstructure data and the soft tissue feature data; the external medical imaging equipment includes at least: photon counting CT equipment or quantitative ultrasound equipment (QUS), musculoskeletal ultrasound equipment (MSUS) or low field MRI magnetic resonance imaging equipment, etc.

[0050] For example, photon counting CT or quantitative ultrasound (QUS) can be used to collect bone microstructure data such as bone density and trabecular thickness, while musculoskeletal ultrasound (MSUS) or low-field MRI can be used to collect soft tissue characteristic data such as the degree of synovial inflammation.

[0051] The feature data acquisition module is also used to acquire the physiological and pathological data through an external detection device interface. The physiological and pathological data includes at least the following core physiological indicators: body temperature, RDW-SD red blood cell distribution width standard deviation, CREA creatinine, CRP (C-reactive protein), ESR erythrocyte sedimentation rate, WBC white blood cell count, PLT platelet count, IL-6 (interleukin-6), IL-8 (interleukin-8), LD lactate dehydrogenase, LYM lymphocyte count, and other indicators, as well as TNF-α (tumor necrosis factor-α) and IL-6 concentrations and core cell distribution information in synovial fluid samples. The aforementioned external detection device interface can be installed on a computer running a medical-specific multimodal large-scale model and is used to connect the feature data acquisition module to an external medical testing system.

[0052] In some preferred embodiments, the decision-making layer, serving as the core intelligent hub, specifically includes: a Western medicine diagnostic module and a Traditional Chinese Medicine (TCM) diagnostic module. These modules are capable of respectively calling upon a medical-specific multimodal large-scale model that has undergone decision-making training to complete the corresponding diagnosis. The Western medicine diagnostic module is used to output Western medicine diagnostic results of subclinical inflammation and micro-erosion volume based on the synovial thickening data, blood flow characteristic data, bone microstructure data and soft tissue characteristic data. The TCM diagnostic module is used to output TCM syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the TCM physical signs data and the physiological and pathological data.

[0053] Furthermore, the TCM diagnostic module specifically includes: a traditional syndrome diagnosis module and a cold / heat syndrome diagnosis module; The traditional syndrome diagnosis module is used to output traditional syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the tongue appearance feature and facial color feature data and the pulse feature data. The cold and heat syndrome diagnosis module is used to output the cold and heat syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the TCM physical signs data and the physiological and pathological data.

[0054] For example, in the actual diagnostic process of the Western medicine diagnostic module, the multimodal large model can establish structure-activity relationships between real-time acquired data such as synovial thickening, blood flow characteristics, bone microstructure, and soft tissue characteristics from MRI, and assess the volume of inflammation and micro-erosion. In the actual diagnostic process of the traditional syndrome diagnosis module, the multimodal large model can establish structure-activity relationships between real-time acquired data such as tongue appearance, facial color, and pulse characteristics, and TCM syndromes such as "qi deficiency and blood stasis" and "yang deficiency and cold coagulation," and make judgments based on the patient's current characteristics. In the actual diagnostic process of the cold and heat syndrome diagnosis module, a high-dimensional matrix can be obtained based on multiple real-time acquired TCM physical signs and physiological and pathological data, and the PCA and OPLS-DA algorithms can be used to classify "cold syndrome" and "heat syndrome."

[0055] It is understood that rheumatoid arthritis (and its comorbidities) syndromes are categorized into various types, including wind-dampness obstruction syndrome, cold-dampness obstruction syndrome, damp-heat obstruction syndrome, phlegm-blood stasis obstruction syndrome, blood stasis obstructing the collaterals syndrome, qi and blood deficiency syndrome, liver and kidney deficiency syndrome, and qi and yin deficiency syndrome. Therefore, the TCM diagnostic module described in this invention not only includes binary classification diagnosis but can also output diagnostic results for the aforementioned multiple TCM syndromes related to rheumatoid arthritis and its comorbidities based on TCM physical signs data and physiological and pathological data. The diagnosis of these multiple syndromes can be trained using a medical-specific multimodal large-scale model with multi-class supervised learning, using historical data related to tongue appearance, complexion, pulse, and physiological and pathological aspects as input, and specific syndrome types as output, to construct a multi-class mapping model; no limitations are imposed here.

[0056] Of course, in the actual application stage at the decision-making level, the specific execution process of the Western medicine diagnosis module and the traditional Chinese medicine diagnosis module is basically the same as the logical framework of the aforementioned medical-specific multimodal large model training stage, and will not be elaborated here.

[0057] Preferably, the decision-making layer further includes: a physiological characteristic scoring module; The physiological feature scoring module is used to perform grayscale scoring on the synovial thickening data, color Doppler scoring on the blood flow feature data, and transmit the corresponding scoring results to the medical-specific multimodal large model for auxiliary diagnosis.

[0058] For example, the physiological characteristic scoring module may specifically include: a grayscale (GS) synovial hypertrophy scoring module, which is used to score the current grayscale (GS) synovial hypertrophy based on grayscale (GS) synovial hypertrophy data; and a color Doppler (CD) blood flow scoring module, which is used to score the current blood flow status based on blood flow data.

[0059] Furthermore, the treatment module includes at least: a drug treatment mode and a physical therapy mode; The drug treatment mode is used to assist in screening drugs from the Chinese medicine database and the Western medicine database based on the diagnosis results of the TCM syndrome and / or the diagnosis results of the Western medicine, and output a personalized drug treatment plan after receiving the doctor's review and confirmation. The drug screening assistance described in this system is intended to provide drug reference information for physicians' reference. This system does not make final medication decisions and does not replace the professional judgment of physicians. Therefore, the drug processing function of this system falls under the category of computer-aided information retrieval and suggestion.

[0060] The physical therapy mode is used to generate personalized massage plans based on the Western medicine diagnosis results of the subclinical inflammation and micro-erosion volume, and to control the dexterous hand to perform traditional Chinese medicine massage therapy.

[0061] Furthermore, the execution process of the physical therapy mode specifically includes: Based on the Western medicine diagnosis results of the subclinical inflammation and micro-erosion volume, and combined with the soft tissue characteristic data, the cartilage damage of the damaged site of the subject to be diagnosed and treated is graded. Based on the location of the injury and the corresponding cartilage injury grading results, the corresponding massage intensity, frequency, duration and technique are retrieved from the preset massage program library to generate the personalized massage program. The personalized massage plan is converted into control commands to control the dexterous hand to perform massage operations on the injured area, and pressure data is fed back in real time to dynamically adjust the massage intensity. After each massage, multimodal data is recollected through the perception layer, and the massage effect is evaluated by the medical-specific multimodal large model. Based on the results of multiple massage evaluations, the operation parameters are iteratively optimized to form a new personalized massage plan and execute subsequent massage operations.

[0062] It is understandable that traditional Chinese massage therapy can have a certain effect on shortening the morning stiffness time and reducing joint swelling and pain in rheumatoid arthritis and its comorbidities. Its techniques start with point massage, while focusing on unblocking the meridians and qi, supplemented by kneading, pressing and pinching techniques.

[0063] Therefore, the TCM massage therapy in this embodiment of the invention is not just mechanically pressed on a single joint, but is implemented along the entire meridian path. It represents a mechanical control method based on data processing results. The purpose is to have the robotic arm drive the dexterous hand to apply physical force to the body surface of the patient according to the pressure parameters output by the model, simulating the massage techniques of a professional physician, so as to realize the closed-loop control process and information processing capability of the system.

[0064] In addition, this system can adapt to the long course and slow changes of rheumatoid arthritis (and its comorbidities), and supports long-term, multiple physical intervention treatments and dynamic monitoring for patients with rheumatoid arthritis (and its comorbidities). Through data collection and feedback analysis before and after each treatment, it can achieve iterative optimization of physical operation parameters.

[0065] It should be noted that this invention does not involve professional disease diagnosis and treatment, but only provides an information processing and automated control scheme based on multimodal data. The output results of this system are all auxiliary references, and the final diagnosis and treatment decision is made by the doctor based on clinical standards.

[0066] To further illustrate the intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities of the present invention, a specific embodiment is provided below, focusing on the static structural configuration and dynamic operational relationship of the perception layer, decision layer, and treatment module. Its execution process includes: First, the perception layer executes.

[0067] First, the specific processing steps of the first feature data acquisition module include: (1) Data on synovial hypertrophy and blood flow characteristics of the patients to be diagnosed and treated were collected using a small musculoskeletal ultrasound probe.

[0068] (2) Data acquisition quality optimization: Set data acquisition thresholds. For example, when the clarity of the acquired synovial hypertrophy data is lower than the preset threshold (grayscale value ≥ 128) and the signal-to-noise ratio of blood flow signal is lower than 30dB, the medical-specific multimodal large model will automatically trigger a re-acquisition command, adjust the acquisition method and angle, until the acquired data reaches the preset qualified data standard.

[0069] Secondly, the specific processing steps of the second feature data acquisition module include: (1) Dexterous hand and sensor deployment: A six-degree-of-freedom dexterous hand is selected, with a depth vision sensor integrated at its wrist and a pressure sensor integrated at its fingertips (measurement range 0-10N, accuracy 0.1N); the dexterous hand and the robotic arm are connected so that the depth vision sensor can be adjusted according to the patient's height and position (sitting / supine position) to ensure that the sensor can be accurately aligned with the patient's face, tongue and wrist pulse.

[0070] (2) Collect tongue and facial color feature data: Environmental control: Set up a standard white light source (color temperature 5500K, brightness 800-1000 lux) in the collection area to avoid strong light and shadow interference. The patient should maintain a natural sitting posture, relax their face, and stick out their tongue naturally (without curling or force) for 3-5 seconds.

[0071] Data Acquisition Operation: The dexterous hand moves the depth vision sensor through a medical-grade multimodal large model to acquire the patient's frontal view, left side view, and right side view (1 frame each), as well as the frontal view and underside view of the tongue (1 frame each). After acquisition, the sensor automatically transmits the image data to the feature extraction module for image denoising and normalization processing, and extracts facial color feature data (redness, gloss) and tongue image feature data (tongue color, tongue coating thickness, tongue coating color), etc.

[0072] (3) Collect pulse characteristic data: Positioning and calibration: The dexterous hand is controlled by a medical-grade multimodal large model to move to the radial artery on the patient's wrist. Pressure feedback is detected in real time by a pressure sensor, and the pressure is adjusted (1-3N) until the radial artery pulsation point is accurately located with a positioning error of no more than 2mm.

[0073] Data Acquisition: The pressure sensor acquires pulse fluctuation signals in real time, for example, with an acquisition duration of 30 seconds and a sampling frequency of 100Hz, and simultaneously records characteristic parameters such as pulse frequency, amplitude, and rhythm. During the acquisition process, the dexterous hand maintains a stable pressure level to avoid pulse signal distortion caused by changes in force.

[0074] Furthermore, the specific processing steps of the feature data acquisition module include: (1) Collect bone microstructure data by quantitative ultrasound equipment or photon counting CT scanning equipment, and collect soft tissue feature data by low field magnetic resonance imaging equipment.

[0075] (2) Simultaneously trigger the interface of the peripheral testing equipment to obtain multiple core physiological indicators such as RDW-SD red blood cell distribution width standard deviation, CREA creatinine, CRP index, ESR erythrocyte sedimentation rate, WBC white blood cell count, PLT platelet count, IL-6 index, IL-8 index, LD lactate dehydrogenase, LYM lymphocyte count, etc.

[0076] Finally, the data is synchronized and stored: all data collected by the first feature data acquisition module, the second feature data acquisition module, and the feature data acquisition module are associated with the basic information of the patient to be diagnosed (such as name, gender, age, and course of disease) and stored in a dedicated database.

[0077] Second, the decision-making level implements the decisions.

[0078] First, the specific processing steps of the physiological characteristic scoring module include: (1) Scoring was performed using the grayscale (GS) synovial thickening scoring module: Scoring criteria were established: Referring to professional guidance documents such as the "Guidelines for Musculoskeletal Ultrasound Examination", a grayscale (GS) synovial hypertrophy scoring standard was developed, which is divided into 0-3 grades; for example, grade 0 (no synovial hypertrophy, synovial thickness <2mm), grade 1 (mild hypertrophy, synovial thickness 2-3mm), grade 2 (moderate hypertrophy, synovial thickness 3-5mm), and grade 3 (severe hypertrophy, synovial thickness >5mm). The grayscale image feature thresholds corresponding to each grade were clearly defined.

[0079] Data preprocessing: The grayscale (GS) synovial thickening data collected by the sensing layer is denoised and enhanced, the synovial boundary is extracted using an edge detection algorithm, and the synovial thickness value is automatically calculated.

[0080] Automatic scoring: The preprocessed synovial thickness value is compared with the scoring standard, and the corresponding scoring result is automatically output by combining the echo intensity (low echo / medium echo / high echo) of the grayscale image. When the scoring result is at two critical values ​​(such as 2mm, 3mm, 5mm), the synovial data of adjacent acquisition points are combined for comprehensive judgment to avoid single numerical errors.

[0081] Scoring verification: The scoring results of each patient can be manually reviewed by two professional physicians. When the multimodal model score is inconsistent with the physician's review score (deviation > level 1), the case data is added to the model training set and retrained to optimize the model's scoring algorithm and ensure that the scoring accuracy is not less than 92%.

[0082] (2) Scoring was performed using the color Doppler (CD) blood flow scoring module: Scoring criteria were established: referencing commonly used clinical blood flow signal scoring criteria, the scores were divided into 0-3 levels: for example, level 0 (no blood flow signal), level 1 (small amount of blood flow signal, 1-2 punctate blood flows), level 2 (moderate blood flow signal, 3-5 punctate blood flows or 1 short linear blood flow), and level 3 (abundant blood flow signal, not less than 6 punctate blood flows or 2 or more linear blood flows), clearly defining the density and morphological thresholds of blood flow signals at each level.

[0083] Blood flow feature data processing: The color Doppler (CD) blood flow feature data collected by the sensing layer is filtered to remove false blood flow signals (such as motion artifacts and noise artifacts) and extract feature parameters such as density, morphology and distribution range of blood flow signals.

[0084] Automatic scoring: Based on the characteristic parameters of blood flow signals, the scoring criteria are matched and the blood flow score results are automatically output. For cases with ambiguous or difficult-to-determine blood flow signals, blood flow data from adjacent collection points are retrieved and combined with the synovial hypertrophy scoring results for comprehensive scoring.

[0085] Scoring calibration: The scoring module is calibrated periodically using data from 100 cases with known blood flow status, and the scoring threshold is adjusted to ensure consistency between the scoring results and the actual clinical diagnosis. For example, a consistency measure index, Kappa value ≥ 0.85, is set.

[0086] Secondly, the specific processing steps of the diagnostic module include: (1) Western Medicine Diagnostic Module: Feature data integration: The synovial hypertrophy data and blood flow feature data collected by the sensing layer, as well as the bone microstructure data (such as volumetric bone mineral density, trabecular bone volume fraction, cortical bone thickness, etc.) collected by the HR-pQCT (high resolution peripheral quantitative computed tomography) equipment and the soft tissue feature data (such as muscle / ligament signal intensity, cartilage thickness, etc.) collected by the MRI (magnetic resonance imaging) equipment are integrated, standardized and normalized, and outliers are removed.

[0087] Diagnostic assessment: Input the above-mentioned synovial thickening data, blood flow characteristic data, bone microstructure data, and soft tissue characteristic data of the subject to be diagnosed into the constructed first structure-activity relationship model, and output the Western medicine diagnosis results of subclinical inflammation degree (mild / moderate / severe) and micro-erosion volume (unit: mm³), and output the diagnostic confidence level. For example, not less than 80% is a valid diagnosis, and less than 80% indicates that additional data should be collected.

[0088] (2) Traditional syndrome diagnosis module: The system receives tongue and facial features, as well as pulse features, from the sensory layer. It then compares these features using a second structure-activity relationship model to calculate the matching degree between the patient's TCM physical signs and various traditional syndromes. The system outputs the syndrome diagnosis result with the highest matching degree as the final traditional syndrome diagnosis result, and also outputs diagnostic criteria, such as "pale purple tongue and choppy pulse, consistent with Qi deficiency and blood stasis syndrome".

[0089] (3) Cold and Heat Syndrome Diagnosis Module: The TCM physical signs and physiological and pathological data of the patients to be diagnosed are input into a pre-trained grouping and classification model based on the OPLS-DA algorithm. The model outputs the diagnosis result of cold and heat syndrome of the current patient's group (cold syndrome / heat syndrome), and also outputs the group confidence score. For example, a confidence score of not less than 85% is considered a valid judgment, and a confidence score of less than 85% is combined with the traditional syndrome diagnosis results for comprehensive judgment.

[0090] Third, the treatment module is executed.

[0091] First, the specific procedures for syndrome determination and pattern selection include: Based on the TCM syndrome diagnosis results from the decision-making level, the system automatically determines the syndrome type of the patient and simultaneously triggers the selection commands for "drug therapy" and "physical therapy". Users can choose a single mode or a combined mode according to the patient's wishes and the severity of their condition.

[0092] Secondly, the specific procedures for drug treatment include: (1) Drug database access: The systemic pharmacology database of traditional Chinese medicine and the relevant Western medicine database are accessed respectively. Drug information in the two databases is extracted, such as the effective components, efficacy, targets and contraindications of traditional Chinese medicine, and the mechanism of action, indications, adverse reactions and drug interactions of Western medicine. A drug information index is established to achieve rapid retrieval.

[0093] (2) Drug screening: Based on the syndrome type, physiological indicators (such as liver and kidney function indicators CREA, inflammatory indicators CRP, etc.), allergy history, etc. of the patient, select drugs that meet the conditions from the drug information index database: Traditional Chinese medicine screening is based on the principle of "differentiation of syndromes and treatment" (such as screening warming and dispelling cold traditional Chinese medicine for cold syndrome, and screening heat-clearing and detoxifying traditional Chinese medicine for heat syndrome); Western medicine screening is based on the treatment guidelines for rheumatoid arthritis (and its comorbidities), and prioritizes immunomodulatory and anti-inflammatory drugs.

[0094] (3) Drug interaction judgment: The drug interaction analysis module in the medical multimodal large model is called to analyze the interactions between various types of traditional Chinese medicine, between traditional Chinese medicine and Western medicine, and between various types of Western medicine. The focus is on detecting whether there are incompatible combinations, drug antagonism, or superposition of toxicity (such as superposition of liver and kidney toxicity). For drug combinations with risks, they are automatically removed and alternative drugs are re-screened.

[0095] (4) Drug recommendation output: Based on the above drug screening results, output a personalized drug treatment plan, including drug name, dosage, usage (oral / topical), course of treatment, and note the precautions for the drug (such as taking it after meals and monitoring liver and kidney function regularly); after the plan is output, it can be reviewed by a professional physician and implemented after confirmation.

[0096] Finally, the specific treatment process of physical therapy includes: (1) Cartilage damage assessment: Based on the Western medicine diagnostic results of subclinical inflammation and micro-erosion volume of the decision-making layer's Western medicine diagnostic module, combined with data such as cartilage thickness and signal intensity collected by MRI magnetic resonance imaging equipment, the degree of cartilage damage of the subject to be diagnosed and treated is graded (mild / moderate / severe), and the location of the damage (such as knee joint, wrist joint) is identified.

[0097] (2) Massage plan generation: Based on the cartilage damage grade and the damaged area, the corresponding massage plan is retrieved from the preset massage plan library, specifying the massage area, massage intensity (mild damage: 1-2N, moderate damage: 2-3N, severe damage: 1-2N, etc.), massage frequency (10-15 times / minute), massage duration (15-20 minutes each time), and massage techniques (such as kneading, pressing, rolling).

[0098] (3) Dexterous hand massage execution: The massage plan is converted into control instructions and transmitted to the dexterous hand, which controls the dexterous hand to move to the injured area and perform the massage operation according to the preset massage force, frequency and technique; During the massage, the pressure sensor at the fingertip of the dexterous hand provides real-time feedback of pressure data, so that the multimodal large model can dynamically adjust the massage force according to the feedback data to avoid excessive force causing secondary damage.

[0099] (4) Monitoring of massage effect: After each massage, the sensory layer re-collects data on synovial thickening, blood flow characteristics, bone microstructure, soft tissue characteristics, and physiological and pathological data of the cartilage injury site of the patient, such as various ultrasound data and physiological indicators, and evaluates the massage effect. Based on the evaluation results, the subsequent massage plan is adjusted. For example, if the injury is reduced, the massage intensity is appropriately increased; if the injury is not improved, the massage technique is adjusted and the treatment course is extended.

[0100] It is understood that the above embodiments are only for ease of understanding and simplification of description, and should not be construed as limitations on the present invention. The present invention does not specifically limit the types of multimodal data, the construction and training methods of medical-specific multimodal large models, the dynamic selection methods of treatment modes, etc.

[0101] Therefore, it can be seen that the embodiments of the present invention can achieve at least one of the following beneficial effects: First, through the perception layer, accurate acquisition of multimodal and multidimensional data is achieved, which improves the standardization of acquisition processes such as synovial thickening and blood flow characteristics and reduces acquisition errors. At the same time, accurate acquisition of physiological characteristics such as tongue appearance, facial color, and pulse avoids the subjectivity and operational deviations of manual acquisition, ensuring the standardization, integrity and reliability of the acquired data, and providing high-quality data support for subsequent diagnosis.

[0102] Secondly, through the decision-making level, precise scoring of physiological characteristics, including gray-scale synovial thickening and color Doppler blood flow, was achieved, with high accuracy and consistency. Through the deep integration of Western medical imaging diagnosis and traditional Chinese medicine syndrome differentiation and treatment, subclinical inflammation and bone micro-erosion volume can be accurately diagnosed, and the degree of lesion can be clarified. At the same time, the objectivity and standardization of traditional Chinese medicine syndromes such as "qi deficiency and blood stasis" and "yang deficiency and cold coagulation" as well as cold and heat syndromes can be achieved, avoiding the subjectivity of traditional Chinese medicine syndrome differentiation and improving the accuracy and repeatability of syndrome diagnosis.

[0103] Third, by combining drug therapy and physical therapy, personalized treatment plans can be intelligently generated. Drug therapy can screen suitable drugs, avoid drug interactions and liver and kidney toxicity, and improve the safety and targeting of medication. Physical therapy can dynamically adjust the massage plan according to the degree of cartilage damage to avoid secondary damage. Thus, a closed-loop diagnosis and treatment system of collection, diagnosis and treatment is formed, which improves the effectiveness and personalization while improving the overall efficiency and quality of diagnosis and treatment.

[0104] In another embodiment of the present invention, a computer device is provided, including at least one processor and at least one memory communicatively connected to said processor; The memory stores instructions that can be executed by the processor to implement the intelligent diagnosis and treatment system for rheumatoid arthritis and its comorbidities using both traditional Chinese and Western medicine, as described in any of the preceding claims.

[0105] It should be noted that all processing procedures described in this system are executed by the aforementioned computer equipment. The core of this invention lies in performing mathematical operations and feature mapping on the collected multimodal data using a medical-specific multimodal large model. The output diagnostic results are only auxiliary information. This system does not directly perform any diagnostic or treatment operations on living human bodies, nor does it output a final clinical diagnostic conclusion. Instead, it provides an information analysis method based on data processing. Therefore, the technical solution of this invention belongs to computer-implemented information processing methods.

[0106] The above system and device embodiments are based on the same principles, and their related aspects can be referenced from each other to achieve the same technical effects. For specific implementation processes, please refer to the foregoing embodiments, which will not be repeated here.

[0107] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0108] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A smart integrated traditional Chinese and Western medicine diagnostic and treatment system for rheumatoid arthritis and its comorbidities, characterized in that, include: The perception layer is used to call a pre-built medical-specific multimodal large model to control the multimodal acquisition module to obtain multimodal data of the object to be diagnosed and treated. The multimodal data includes at least: synovial hypertrophy data, blood flow characteristic data, bone microstructure data, soft tissue characteristic data, traditional Chinese medicine signs data, and physiological and pathological data. The decision layer receives multimodal data collected by the perception layer and invokes the medical-specific multimodal large model to perform integrated traditional Chinese and Western medicine diagnostic operations for rheumatoid arthritis and its comorbidities, outputting corresponding traditional Chinese medicine syndrome diagnosis results and / or Western medicine diagnosis results; and The treatment module is used to receive the diagnostic results output by the decision layer, dynamically select the treatment mode that matches the patient, and provide a corresponding rehabilitation treatment plan.

2. The intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to claim 1, characterized in that, The multimodal acquisition module includes at least: The first feature data acquisition module is used to acquire the synovial thickening data and the blood flow feature data through the first multimodal sensor group configured by the dexterous hand; The second feature data acquisition module is used to acquire the TCM vital signs data through the second multimodal sensor group configured by the dexterous hand; The feature data acquisition module is used to connect to external medical imaging equipment and acquire the bone microstructure data and the soft tissue feature data; it is also used to acquire the physiological and pathological data through the interface of peripheral detection equipment.

3. The intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to claim 2, characterized in that, The TCM physical signs data include at least: tongue appearance characteristics and facial color characteristics data, and pulse characteristics data; The physiological and pathological data include at least: body temperature, RDW-SD standard deviation of red blood cell distribution width, CREA creatinine, CRP index, ESR erythrocyte sedimentation rate, WBC white blood cell count, PLT platelet count, IL-6 index, IL-8 index, LD lactate dehydrogenase, and LYM lymphocyte count.

4. The intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to claim 2, characterized in that, The training tasks of the medical-specific multimodal large model in the perception phase include: Based on expert detection operation samples and annotations of patients with rheumatoid arthritis and their comorbid history, the multimodal data obtained by controlling the multimodal acquisition module to perform the multimodal data acquisition operation is used as the training target, and a supervised learning algorithm is used to iteratively train the medical-specific multimodal large model.

5. The intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to claim 3, characterized in that, The training tasks of the medical-specific multimodal large model during the decision-making phase include: Based on historical data of synovial thickening, blood flow characteristics, bone microstructure, and soft tissue characteristics, the subclinical inflammation degree and microerosion volume of rheumatoid arthritis and its comorbidities are used as output labels. An LSTM long short-term memory network is used to construct and train the first structure-activity relationship model so that the medical-specific multimodal large model has the ability to diagnose subclinical inflammation and microerosion volume. Based on historical data of TCM physical signs, and using traditional TCM syndromes as output labels, a second structure-activity relationship model is constructed and trained using a supervised learning algorithm, so that the medical-specific multimodal large model has the ability to diagnose traditional syndromes. Based on historical data of TCM physical signs and physiological and pathological data, cold syndrome and heat syndrome are used as group labels. PCA principal component analysis and OPLS-DA orthogonal partial least squares discriminant analysis algorithms are used to construct and train a group classification model so that the medical-specific multimodal large model has the ability to diagnose cold and heat syndromes.

6. The intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to claim 5, characterized in that, The decision-making layer includes a Western medicine diagnostic module and a Traditional Chinese Medicine (TCM) diagnostic module. These modules can respectively invoke a medical-specific multimodal large-scale model that has undergone decision-making training to complete the corresponding diagnosis. The Western medicine diagnostic module is used to output Western medicine diagnostic results of subclinical inflammation and micro-erosion volume based on the synovial thickening data, blood flow characteristic data, bone microstructure data and soft tissue characteristic data. The TCM diagnostic module is used to output TCM syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the TCM physical signs data and the physiological and pathological data.

7. The intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to claim 6, characterized in that, The TCM diagnostic module specifically includes: a traditional syndrome diagnosis module and a cold-heat syndrome diagnosis module; The traditional syndrome diagnosis module is used to output traditional syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the tongue appearance feature and facial color feature data and the pulse feature data. The cold and heat syndrome diagnosis module is used to output the cold and heat syndrome diagnosis results for rheumatoid arthritis and its comorbidities based on the TCM physical signs data and the physiological and pathological data.

8. The intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to claim 6, characterized in that, The treatment module includes at least: a drug treatment mode and a physical therapy mode; The drug treatment mode is used to assist in screening drugs from the Chinese medicine database and the Western medicine database based on the diagnosis results of the TCM syndrome and / or the diagnosis results of the Western medicine, and output a personalized drug treatment plan after receiving the doctor's review and confirmation. The physical therapy mode is used to generate personalized massage plans based on the Western medicine diagnosis results of the subclinical inflammation and micro-erosion volume, and to control the dexterous hand to perform traditional Chinese medicine massage therapy.

9. The intelligent integrated traditional Chinese and Western medicine diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to claim 8, characterized in that, The execution process of the physical therapy mode specifically includes: Based on the Western medicine diagnosis results of the subclinical inflammation and micro-erosion volume, and combined with the soft tissue characteristic data, the cartilage damage of the damaged site of the subject to be diagnosed and treated is graded. Based on the location of the injury and the corresponding cartilage injury grading results, the corresponding massage intensity, frequency, duration and technique are retrieved from the preset massage program library to generate the personalized massage program. The personalized massage plan is converted into control commands to control the dexterous hand to perform massage operations on the injured area, and pressure data is fed back in real time to dynamically adjust the massage intensity. After each massage, multimodal data is recollected through the perception layer, and the massage effect is evaluated by the medical-specific multimodal large model. Based on the results of multiple massage evaluations, the operation parameters are iteratively optimized to form a new personalized massage plan and execute subsequent massage operations.

10. A computer device, characterized in that, It includes at least one processor and at least one memory communicatively connected to the processor; The memory stores instructions that can be executed by the processor to implement the intelligent diagnosis and treatment system for rheumatoid arthritis and its comorbidities according to any one of claims 1 to 9.

Images