A traditional Chinese medicine gynecological syndrome intelligent identification method and system based on multi-modal sign fusion

Abstract

The application discloses a traditional Chinese medicine gynecological syndrome intelligent identification method and system based on multi-modal sign fusion, and relates to the technical field of syndrome identification. The method comprises the following steps: pre-constructing a syndrome knowledge constraint graph, wherein the graph comprises initial weights of syndrome contribution of different gynecological diseases and different physiological stages and core correlation features corresponding to each traditional Chinese medicine gynecological syndrome, and obtaining a knowledge graph item; and collecting multi-modal sign data of an object to be identified in real time to obtain a real-time collection item. The application pre-constructs a syndrome knowledge constraint graph, and rules of disease type staging modal initial weight and syndrome feature correlation priority are used to avoid the problem that a pure AI model deviates from traditional Chinese medicine logic from the root, for example, differentiating modal weights for premenstrual menstrual disease and non-menstrual leukorrhea, and clearly sorting core and secondary features of a syndrome, so that physiological cycle interference and feature weight homogenization are effectively avoided.

Description

Technical Field

[0001] This invention relates to the field of syndrome identification technology, specifically to a method and system for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion. Background Technology

[0002] Syndrome identification is a core concept in Traditional Chinese Medicine (TCM). It refers to the process of determining the current "syndrome" of a patient based on their symptoms and signs. TCM gynecological syndrome identification is a unique application of the TCM diagnostic and treatment system in gynecological care. Its core is the process of determining the syndrome attributes of gynecological diseases under the guidance of TCM's holistic and diagnostic perspectives, combined with the unique physiological and pathological characteristics of women. Currently, commonly used TCM gynecological syndrome identification methods are mainly divided into two categories: traditional classical methods and modern innovative auxiliary methods. Traditional classical methods are adapted to the general diagnostic system, using the Eight Principles, Zang-Fu organs, and Qi, Blood, and Body Fluids as a general foundation, adjusting the emphasis according to gynecological pathological characteristics. Modern innovative auxiliary identification methods, such as syndrome rating scales, develop standardized syndrome scoring scales for specific gynecological diseases, such as the Polycystic Ovary Syndrome Syndrome Scale and the Menopausal Syndrome TCM Syndrome Scoring Scale. Symptoms are scored according to severity, and reaching the corresponding threshold determines the syndrome type. These are mostly used in clinical research and as auxiliary diagnostic tools for primary care physicians.

[0003] The patent publication number CN121598178A discloses a method and system for identifying Yin-Yang syndromes in Traditional Chinese Medicine based on the fusion of bioelectrical signal features. This method objectively collects bioelectrical signals through remote photoplethysmography pulse waves, models constrained neural networks based on the Yin-Yang theory of Traditional Chinese Medicine, and adopts an intelligent decision-making mechanism that uses multi-model confidence weighted fusion. This eliminates the subjectivity and experience dependence in traditional Chinese Medicine syndrome identification, achieves a deep integration of the constraints of Traditional Chinese Medicine theory and modern machine learning technology, and improves the objectivity of Yin-Yang syndrome identification.

[0004] When using the above-mentioned and similar technical solutions for syndrome differentiation, the contribution of each modality of signs to syndrome differentiation varies depending on the gynecological disease and the physiological cycle, resulting in low accuracy. For example, during the menstrual cycle, the manifestations of signs such as tongue appearance and pulse change with hormone levels. Furthermore, different gynecological diseases, such as uterine fibroids and pelvic inflammatory disease, have different emphases in their signs. Therefore, conventionally treating all signs equally or using a fixed-weight syndrome differentiation method is difficult to adapt to complex and ever-changing clinical situations, ultimately affecting the accuracy and reliability of syndrome differentiation. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion, comprising:

[0007] A pre-constructed knowledge graph of syndromes is constructed. The graph includes the initial weights of the syndrome differentiation contributions of different gynecological diseases and different physiological stages, as well as the core association features corresponding to each TCM gynecological syndrome, to obtain knowledge graph items.

[0008] Multimodal vital signs data of the object to be identified are collected in real time to obtain real-time collection items. Based on the real-time collection items, the modal data are standardized and feature-encoded to obtain a single-modal feature vector set.

[0009] By matching single-modal feature vector sets with knowledge graph items, missing modal data is obtained, and missing modal items are obtained.

[0010] Based on knowledge graph items, a two-stage dynamic weight adjustment strategy is adopted to complete multimodal feature fusion. The first stage weight adjustment is based on real-time collected items to obtain the first stage adjusted weight items.

[0011] The second-stage weight adjustment calculates the correlation between each effective modal feature and the target TCM gynecological syndrome through an attention mechanism, and performs a second fine-tuning on the first-stage adjustment weights to obtain the second-stage adjustment weight items.

[0012] Using the second-stage adjusted weight term as input data, the pre-trained syndrome classification layer is input to obtain preliminary syndrome diagnosis results. The confidence of the diagnosis results is calculated by combining the completeness of the current input modality and the correlation between features and syndromes, resulting in a result calculation term. A preset threshold is set, and the identification result is output based on the matching result of the result calculation term and the preset threshold, thus achieving intelligent identification of gynecological syndromes by integrating multimodal data.

[0013] Furthermore, the syndrome knowledge constraint graph incorporates two types of rules, and the methods for obtaining knowledge graph items include:

[0014] The first type of rule in the syndrome knowledge constraint graph is the initial weight rule of disease stage modality. For each type of gynecological disease and each type of female physiological stage, different modal signs are pre-set with initial weights for their diagnostic contributions, resulting in initial diagnostic weight terms.

[0015] The first type of rule in the syndrome knowledge constraint graph is the syndrome feature association priority rule, which pre-determines the priority ranking of the core association features corresponding to each TCM gynecological syndrome, and obtains the syndrome ranking item;

[0016] The initial dialectical weight term and the syndrome ranking term are combined to obtain the knowledge graph term.

[0017] Furthermore, the multimodal physical examination data includes subjective symptom time-series modal data, objective modal data of the four diagnostic methods, gynecological examination modal data, and gynecological imaging modal data. The methods for acquiring real-time data include:

[0018] Subjective symptom temporal modal data includes menstrual history, marital and reproductive history, clinical subjective symptoms, and disease course information of the subject to be identified; objective modal data of the four diagnostic methods includes standardized collection of tongue, pulse, and facial images; gynecological laboratory modal data includes numerical test results of sex hormones, anti-Müllerian hormones, vaginal microecology, inflammatory factors, and coagulation function; and gynecological imaging modal data includes pelvic ultrasound and MRI imaging data.

[0019] Furthermore, the method for obtaining the single-modal feature vector set includes:

[0020] Based on the subjective symptom temporal modal data, a pre-trained TCM gynecology-specific word vector model was used to complete feature encoding. Based on the objective modal data of the four diagnostic methods, a lightweight convolutional neural network model was used to extract quantitative features to complete encoding. Based on the gynecological examination modal data, normalization processing was used to complete encoding. Based on the gynecological imaging modal data, a pre-trained medical imaging feature extraction model was used to extract lesion omics and blood perfusion features to complete encoding.

[0021] Furthermore, the method for obtaining the missing modal term includes:

[0022] The system obtains the number of categories in the single-modality feature vector set to get real-time category items. Based on the real-time category items, it performs category matching with knowledge graph items to determine whether there are missing categories in the single-modality feature vector set. When there are missing categories, it obtains the number of missing categories and the missing category feature data corresponding to the number of missing categories, and then obtains the modality missing items.

[0023] Furthermore, the method for obtaining the weight adjustment term in the first stage includes:

[0024] Based on the knowledge graph item, the initial weights corresponding to the single-modal feature vector set are retrieved from the knowledge graph item to obtain the weight item;

[0025] Based on the obtained weight terms, the obtained weight terms are adaptively adjusted in the first round according to the missing modality terms. The initial weights of the missing modality terms are proportionally allocated to the corresponding features of the effective modalities of the same dimension to obtain the first-stage adjusted weight terms.

[0026] Furthermore, the method for obtaining the second-stage adjustment weight term includes:

[0027] Based on the first stage of adjusting the weights, candidate syndromes are pre-screened, a set of candidate syndromes is output, and low-probability syndromes are excluded.

[0028] Based on the single-modal feature vector set, the correlation score between each modality and the candidate syndrome set is calculated to obtain the modality correlation score item;

[0029] Generate weight fine-tuning coefficients, and then perform secondary fine-tuning on the first-stage adjusted weights based on the combination of modal correlation score items and weight fine-tuning coefficients to obtain the second-stage adjusted weight items.

[0030] Furthermore, the method for obtaining the result calculation item includes:

[0031] Based on the second-stage adjusted weight set and modal missing items, the modal completeness score is calculated to obtain the first calculation item;

[0032] The matching score of syndrome features is calculated based on knowledge graph items, including core feature hit, secondary feature hit and conflict feature deduction, to obtain the second calculation item;

[0033] The final confidence score is generated by fusing the first and second calculation items, and confidence levels and corresponding actions are set to obtain the result calculation items.

[0034] Furthermore, the method for obtaining the result output items includes:

[0035] When the calculated result exceeds the preset threshold, the syndrome diagnosis result and the contribution explanation of each modality feature are output. When the confidence level does not exceed the preset threshold, the feedback of supplementary sampling of missing modalities is pushed.

[0036] A TCM gynecological syndrome intelligent identification system based on multimodal physical sign fusion, which uses the aforementioned TCM gynecological syndrome intelligent identification method based on multimodal physical sign fusion, includes:

[0037] The graph construction module pre-constructs a syndrome knowledge constraint graph, which includes the initial weights of the syndrome differentiation contributions of different gynecological diseases and different physiological stages, as well as the core association features corresponding to each TCM gynecological syndrome, to obtain knowledge graph items;

[0038] Data acquisition module: Real-time acquisition of multimodal vital sign data of the object to be identified, obtaining real-time acquisition items, and standardized feature encoding of each modal data based on the real-time acquisition items to obtain a single-modal feature vector set;

[0039] Weight adjustment module: Based on the matching of single-modal feature vector set and knowledge graph items, missing modal data is obtained to obtain missing modal items. Based on knowledge graph items, a two-stage dynamic weight adjustment strategy is adopted to complete multimodal feature fusion. The first stage weight adjustment is based on real-time collected items to obtain the first stage adjusted weight items. The second stage weight adjustment calculates the correlation between each effective modal feature and the target TCM gynecological syndrome through the attention mechanism, and performs a second fine-tuning on the first stage adjusted weights to obtain the second stage adjusted weight items.

[0040] The result output module uses the second-stage adjusted weight term as input data to input the pre-trained syndrome classification layer to obtain preliminary syndrome diagnosis results. It calculates the confidence of the diagnosis results by combining the completeness of the current input modality and the correlation between features and syndromes, and obtains the result calculation term. It sets a preset threshold and outputs the identification result based on the matching result of the result calculation term and the preset threshold, thus obtaining the result output term.

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] This intelligent identification method and system for TCM gynecological syndromes based on multimodal symptom fusion avoids the problem of pure AI models deviating from TCM logic by pre-constructing a syndrome knowledge constraint graph and using two types of rules: initial weights of disease stage modalities and priority of syndrome feature associations. For example, it sets differentiated modal weights for premenstrual syndromes and non-menstrual syndromes of leukorrhea, and clarifies the ranking of core and secondary features of syndromes, effectively avoiding interference from the menstrual cycle and homogenization of feature weights.

[0043] Meanwhile, the multi-scenario adaptable, multi-dimensional data collection covers four modalities: subjective symptom time series, objectification of the four diagnostic methods, gynecological examination, and imaging data. It is adapted to three scenarios: tertiary hospitals, primary care, and home care. Through standardized collection, AI correction, and system integration, it solves the problems of data accessibility and standardization in different scenarios, and eliminates time series misalignment and invalid data interference.

[0044] Furthermore, the personalized fusion of dual-stage dynamic weights, through the first stage of weight supplementation based on modal missingness and the second stage of individual feature fine-tuning based on attention mechanism, ensures the rigor of the diagnostic logic under missing modalities while accurately matching the differences in the core characteristics of individual patients. The interpretable confidence level output, combined with modal completeness and syndrome feature matching degree to calculate confidence, provides explanations of feature contribution for high confidence results and pushes precise supplementary sampling feedback for low confidence results. This not only solves the black box problem of AI diagnosis but also improves the reliability of diagnosis through the supplementary sampling mechanism. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the knowledge graph construction process of the present invention;

[0046] Figure 2 This is a schematic diagram of the multimodal acquisition process of the present invention;

[0047] Figure 3 This is a schematic diagram of the two-stage fusion process of the present invention;

[0048] Figure 4 This is a schematic diagram of the confidence calculation process of the present invention;

[0049] Figure 5This is a schematic diagram of the result output process of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] In the diagnosis and treatment of gynecological diseases, syndrome differentiation is a crucial step, directly impacting the formulation of subsequent treatment plans and the evaluation of efficacy. However, due to the complexity of gynecological diseases and the dynamic nature of the menstrual cycle, the contribution of different modalities of physical signs in the differentiation process varies significantly. First, during the menstrual cycle, fluctuations in hormone levels cause a series of physiological changes, which are reflected in physical signs such as tongue appearance and pulse. For example, before and after menstruation, the tongue coating may become thinner, and the pulse may become relatively slippery and rapid. If the diagnostician ignores these physiological cycle influences and simply attributes these changes to pathological factors, misdiagnosis may occur. Furthermore, the clinical manifestations of different gynecological diseases have different focuses. Taking uterine fibroids and pelvic inflammatory disease as examples, the main symptoms of uterine fibroids may include increased menstrual flow and prolonged menstrual periods, while pelvic inflammatory disease may manifest as lower abdominal pain and fever. If all physical signs are treated equally, or if a fixed-weight differentiation method is used, it will be difficult to capture the unique clinical features of these diseases, thus affecting the diagnosis. The accuracy of the diagnosis is crucial. The technical solution provided in this application, by pre-constructing a syndrome knowledge constraint graph and using two types of rules—initial weights for disease staging modalities and priority of syndrome feature associations—fundamentally avoids the problem of pure AI models deviating from traditional Chinese medicine logic. For example, it sets differentiated modal weights for premenstrual syndromes and non-menstrual syndromes of leukorrhea, clearly defining the ranking of core and secondary syndrome features, effectively avoiding interference from the menstrual cycle and homogenization of feature weights. Multi-scenario adaptable, multi-dimensional data collection covers four modalities: subjective symptom time-series, objectification of the four diagnostic methods, gynecological examination, and imaging data. It adapts to three scenarios: tertiary hospitals, primary care, and home care. Through standardized collection, AI correction, and system integration, it solves the problems of data accessibility and standardization in different scenarios, eliminating temporal misalignment and invalid data interference. The personalized fusion of two-stage dynamic weights, through the first stage of weight supplementation based on modal missingness and the second stage of individual feature fine-tuning based on attention mechanisms, ensures both the rigor of the diagnostic logic under missing modalities and accurate matching of individual patient core feature differences. Figures 1-5 As shown, it includes steps S100-S800.

[0052] Step S100: Preconstruct a syndrome knowledge constraint graph. The graph includes the initial weights of the syndrome differentiation contributions of different gynecological diseases and different physiological stages, as well as the core association features corresponding to each TCM gynecological syndrome, to obtain knowledge graph items.

[0053] It is important to note that, such as Figure 1 As shown, the syndrome knowledge constraint graph has two types of built-in rules. The methods for obtaining knowledge graph items include: the first type of rule in the syndrome knowledge constraint graph is the initial weight rule for disease staging modality, which pre-sets the initial weight of the syndrome contribution of different modal signs for each type of gynecological disease and each type of female physiological stage, thus obtaining the initial syndrome weight item; the second type of rule in the syndrome knowledge constraint graph is the priority rule for syndrome feature association, which pre-labels the priority ranking of the core association features corresponding to each TCM gynecological syndrome, thus obtaining the syndrome ranking item; the initial syndrome weight item and the syndrome ranking item are combined to obtain the knowledge graph item.

[0054] Specifically, the initial weighting rules for disease staging modalities were constructed by obtaining optimal values ​​through large-sample case statistics. For example, for the premenstrual stage of menstrual disorders, diagnostic modalities such as sex hormones / AMH contribute the most to the diagnosis, so the initial weights were set at 0.35-0.45. The weights for objective modalities based on the four diagnostic methods were 0.25-0.35, and the weights for subjective symptom modalities were 0.25-0.35. For the non-menstrual stage of leukorrhea, the characteristics of leukorrhea and vaginal microecology are the core diagnostic criteria, so the initial weights for this type of modality were set at 0.45-0.55. The other two modal weights each account for 0.2-0.3. In the syndrome-feature association priority rule, each syndrome is clearly marked with 3-5 core features and 6-10 secondary features. For example, the core feature priority of liver qi stagnation syndrome is: wiry pulse > premenstrual breast distension > LH / FSH ratio ≥2.5 > red tongue edge > emotional irritability. This avoids low-priority features interfering with the core syndrome differentiation judgment. This map is a hard constraint for all subsequent weight adjustments and result verification, fundamentally solving the pain point that the syndrome differentiation results of pure AI models do not conform to the clinical logic of traditional Chinese medicine.

[0055] Step S200: Collect multimodal vital sign data of the object to be identified in real time to obtain real-time collection items.

[0056] It is important to note that multimodal physical examination data includes subjective symptom temporal modality data, objective modality data of the four diagnostic methods, gynecological laboratory modality data, and gynecological imaging modality data. The methods for acquiring real-time data include: subjective symptom temporal modality data includes the menstrual history, marital and reproductive history, clinical subjective symptoms, and disease course information of the subject to be identified; objective modality data of the four diagnostic methods includes standardized tongue, pulse, and facial images; gynecological laboratory modality data includes numerical test results of sex hormones, anti-Müllerian hormones, vaginal microecology, inflammatory factors, and coagulation function; and gynecological imaging modality data includes pelvic ultrasound and MRI imaging data.

[0057] Specifically, the four types of multimodal data cover all dimensions of information sources for TCM gynecological diagnosis, and are adapted to three core application scenarios: tertiary hospitals, primary healthcare centers, home care, and the internet. Subjective symptom temporal modal data can be connected to the structured electronic medical records of the HIS system in tertiary hospital scenarios. In primary care and home care scenarios, standardized check-in questionnaires are used for collection. The questionnaires are divided into four menstrual phases: postmenstrual, intermenstrual, premenstrual, and menstrual, with time-series anchor points set to clearly mark the physiological stage of symptom appearance and avoid diagnostic errors caused by temporal misalignment. The objective modal data of the four diagnostic methods are collected by tertiary hospitals using standardized tongue imaging instruments with color difference correction and pressure pulse diagnosis instruments. Data collection utilizes AI-corrected data collection functions on mobile devices in grassroots and home settings, automatically filtering out invalid data such as dietary staining and abnormal pulse waves after exercise. Gynecological laboratory modal data is directly connected to the hospital's LIS system, or OCR is used to identify and verify user-uploaded laboratory reports, automatically matching the reference range for the user's menstrual cycle. Gynecological imaging modal data is connected to the original pelvic ultrasound and MRI images from the hospital's PACS system, excluding invalid images with blurriness or angular deviations. These four types of data, from subjective to objective and from non-invasive to invasive, cover all information needs for clinical diagnosis and adapt to the data accessibility of different scenarios.

[0058] Step S300: Based on the real-time acquisition items, perform standardized feature encoding on each modal data to obtain a single-modal feature vector set.

[0059] It is important to note that, such as Figure 2 As shown, the methods for obtaining single-modal feature vector sets include: using a pre-trained TCM gynecology-specific word vector model to complete feature encoding based on subjective symptom time-series modal data; using a lightweight convolutional neural network model to extract quantitative features based on objective four diagnostic methods modal data; using normalization processing to complete encoding based on gynecological examination modal data; and using a pre-trained medical image feature extraction model to extract lesion omics and blood perfusion features based on gynecological image modal data.

[0060] Specifically, for subjective symptom time-series modal data, the data collection scenarios were first defined, including outpatient clinics in tertiary hospitals, primary care / public health settings, and home / internet-based scenarios. Data from outpatient clinics in tertiary hospitals came from structured electronic medical records in the hospital's HIS system and physician specialist consultation entries. The collection criteria strictly matched the symptom terminology in the *Guidelines for Clinical Diagnosis and Treatment of Gynecology in Traditional Chinese Medicine*, and data was entered in a structured format according to symptom + severity + time of occurrence. Data from primary care / public health settings came from the community health service center's public health reporting system and standardized consultation questionnaires for village doctors. The collection criteria were based on… The questionnaire uses checkboxes to avoid bias from free text. Data for home / internet scenarios comes from questionnaires specific to gynecological health mini-programs / apps and user-uploaded symptom records. The collection standard is to set time-series anchors in the questionnaire, such as clearly marking which stage of the four menstrual cycles (postmenstrual / intermenstrual / premenstrual / menstrual) the symptoms belong to. Then, core data is collected, including menstrual history (age of menarche, cycle length, menstrual duration, menstrual flow grade <1-5 corresponding to daily / nighttime sanitary napkin usage>, menstrual color, menstrual texture, and dysmenorrhea severity), and marital and reproductive history (parity, number of pregnancies, number of deliveries, etc.). The system automatically extracts three core information categories: symptom entities, severity labels, and time-series labels. This is achieved by considering factors such as history of miscarriage, contraceptive methods, systemic / local symptoms (breast tenderness, lower back and knee weakness, mood swings, vaginal discharge characteristics, etc.), and duration of illness. For free text (such as handwritten medical records or user-generated descriptions), a pre-trained Traditional Chinese Medicine (TCM) gynecology named entity recognition (NER) model is used to automatically extract these three categories of core information from the text. Non-standard terms such as "heavy menstrual flow" and "cold lower abdomen" are mapped to the 1200 standard terms in the *TCM Gynecology Syndrome Diagnosis Standards*, eliminating discrepancies in expression. The pre-trained model then... The BERT-small word vector model, specifically designed for gynecology, has been successfully encoded. The model's pre-training corpus includes textbooks on traditional Chinese medicine gynecology, 500,000 electronic gynecological medical records, ancient gynecological books from various dynasties, and 20 national-level gynecological diagnosis and treatment guidelines. After 100 rounds of fine-tuning, a 128-dimensional standard word vector is output. An additional 8-dimensional positional encoding is added to the temporal features to mark which time step of the current menstrual cycle the symptom belongs to. The word vectors and positional encodings are concatenated to finally output a 136-dimensional fixed-length subjective symptom temporal feature vector, with missing fields set to 0 at their corresponding feature positions.

[0061] Specifically, for the objective modal data of the four diagnostic methods, a constitution type is defined, including tongue appearance, pulse appearance, and facial features. Tongue appearance is obtained using a standardized tongue image acquisition device with a ring-shaped cold light source, 12-megapixel resolution, and color difference correction module, or via a mobile AI tongue image acquisition app. The tongue is naturally extended in a resting state, without food staining or denture obstruction, and three images are collected and averaged. Pulse appearance is obtained using a wrist pressure pulse diagnostic device with three thin-film pressure sensors and a sampling rate of 1000Hz, or via a wearable pulse diagnosis wristband, after sitting quietly for 5 minutes. The pulse wave of the right hand was collected for 30 seconds at a time, excluding interference from movement and emotional fluctuations. Facial images were collected using a facial image acquisition device with face correction and AI correction via the front-facing camera of a mobile phone. Under conditions of no makeup and no direct strong light, skin color features of the forehead, cheekbones, and lips were collected. Tongue image features were first encoded using a finely tuned MobileNetV2 lightweight convolutional neural network. The training set consisted of 100,000 tongue images labeled by gynecologists, with labels including tongue color, coating color, coating thickness, tongue shape, petechiae / teeth marks, etc. Sixteen-dimensional quantitative features were extracted, including 8 dimensions of color features (mean values ​​of RGB and Lab color spaces for tongue and tongue coating) and 8 dimensions of morphological features (tongue enlargement, teeth marks, petechiae score, coating thickness, and coating greasiness score). Pulse feature encoding was then performed. First, wavelet denoising was applied to the original pulse signal to remove power frequency interference and motion artifacts. A finely tuned ShuffleNet lightweight convolutional neural network was used, with a training set of 80,000 labeled gynecological pulse data points, labeled with terms such as wiry, slippery, thin, and deep pulse. The system uses 12-dimensional time-domain and frequency-domain features, including pulse rate, main wave amplitude, tidal wave amplitude, descending isthmus height, and pulse strength score. Finally, facial feature encoding is performed. First, facial key points are aligned, and the forehead, cheekbone, and lip regions are cropped to extract 8-dimensional quantitative features: facial red, yellow, and blue color scores, lip color scores, and chloasma severity scores. Finally, the features are concatenated and output, combining the tongue, pulse, and facial features to output a 36-dimensional fixed-length objective feature vector for the four diagnostic methods. The feature positions corresponding to uncollected signs are set to 0.

[0062] Specifically, for gynecological laboratory modal data, all data is connected to the hospital's LIS system or uploaded by users after OCR recognition and manual verification. The core laboratory test items and reference standards are shown in Table 1.

[0063] Table 1

[0064]

[0065] First, data cleaning and derivative feature calculation are performed to remove outliers, such as extreme values ​​exceeding the range of the testing instrument or units with inconsistent units, which are then converted. Gynecological-specific derivative features are calculated, such as the FSH / LH ratio, LH peak offset, and AMH age correction value, totaling 5 derivative features. This supplements the diagnostic information not covered by the original test items. All 24 original test items and derivative features are subjected to Min-Max normalization and mapped to the [0,1] interval. Features exceeding the upper limit of the reference range are set to 1, and those below the lower limit are set to 0. Features within the interval are mapped proportionally. Finally, a 24-dimensional fixed-length gynecological test feature vector is output, with the feature positions of undetected test items set to 0.

[0066] For gynecological imaging modal data, all data are sourced from the hospital's PACS system, consisting of original images or screenshots of image reports annotated by physicians. The core image types and annotation rules are shown in Table 2.

[0067] Table 2

[0068]

[0069] First, lesion region segmentation was performed using a finely tuned U-Net medical image segmentation model. The training set consisted of 30,000 labeled gynecological pelvic images, which automatically segmented the ovarian, uterine, and lesion regions without requiring manual secondary annotation. Next, omics feature extraction was performed using a finely tuned ResNet-18 medical image feature extraction model. The training set consisted of 30,000 labeled gynecological pelvic images. 18-dimensional quantitative features were extracted from the segmented regions, including ovarian volume, AFC, endometrial thickness, fibroid size, blood perfusion parameters, and pelvic effusion volume. All extracted omics features were Min-Max normalized and mapped to the [0,1] interval. Finally, an 18-dimensional fixed-length gynecological image feature vector was output, with the corresponding feature positions for which no imaging examination was performed set to 0.

[0070] Step S400: Match the single-modal feature vector set with the knowledge graph items to obtain the missing modal data and obtain the missing modal items.

[0071] It should be noted that the method for obtaining modal missing items includes: obtaining the number of categories in the single modal feature vector set to obtain real-time category items; performing category matching based on the real-time category items and knowledge graph items to determine whether there are missing categories in the single modal feature vector set; when there are missing categories, obtaining the number of missing categories and the missing category feature data corresponding to the number of missing categories, and then obtaining the modal missing items.

[0072] Specifically, firstly, a unique identifier is pre-assigned to each of the four modalities: subjective symptoms = bit 1, four diagnostic methods = bit 2, laboratory tests = bit 3, and imaging = bit 4. The real-time collected single-modal feature vector set is traversed. If a valid feature vector exists for a certain modality, the corresponding identifier is assigned a value of 1; otherwise, it is assigned a value of 0. A 4-bit presence mask is generated as the core carrier of the missing modality item. This step does not only count the number of missing modalities but also simultaneously marks the type of missing modality, providing a basis for the subsequent first-stage adaptive weight adjustment. At the same time, the corresponding positions of the feature vectors of all missing modalities are uniformly set to 0 to ensure that the dimensions of all single-modal feature vectors are uniform and there will be no dimension mismatch problem. For example, if a user in a home scenario only uploads subjective symptoms and tongue image data but does not provide laboratory tests or imaging data, the presence mask is [1,1,0,0], and the corresponding missing modalities are laboratory tests and imaging.

[0073] Step S500: Based on the knowledge graph items, a two-stage dynamic weight adjustment strategy is adopted to complete the multimodal feature fusion. The first stage weight adjustment is based on the real-time collected items to obtain the first stage adjusted weight items.

[0074] It is important to note that, such as Figure 3 As shown, the method for obtaining the first-stage adjusted weight terms includes: based on the knowledge graph terms, retrieving the initial weights corresponding to the single-modality feature vector set from the knowledge graph terms to obtain the obtained weight terms; based on the obtained weight terms, performing the first round of adaptive adjustment on the obtained weight terms according to the missing modality terms, and proportionally allocating the initial weights of the missing modality terms to the corresponding features of the effective modalities of the same dimension to obtain the first-stage adjusted weight terms.

[0075] Specifically, the first-stage weight adjustment strictly follows the logic of TCM syndrome differentiation to avoid information loss caused by missing modalities. First, based on the disease label and current physiological stage of the object to be identified, the initial weights corresponding to the four modalities are retrieved from the knowledge graph. Then, the syndrome differentiation label of the missing modalities is matched, and the initial weights of the missing modalities are preferentially allocated to the feature subsets of effective modalities with a correlation of ≥70% with the syndrome differentiation. If there is no corresponding feature subset, the weights are allocated proportionally according to the initial weights of the effective modalities. For example, if a patient with premenstrual PCOS has a missing test modality and the syndrome differentiation is kidney deficiency / liver stagnation, the initial weight of the test modality is 0.4, which is preferentially allocated to the feature subsets of the four diagnostic methods and subjective symptoms that correspond to kidney deficiency and liver stagnation, such as pale tongue, soreness and weakness of the waist and knees, and wiry pulse, rather than being evenly allocated to all effective features. This ensures that even if the test modality is missing, the weight allocation still conforms to the logic of TCM syndrome differentiation, and the accuracy is improved by more than 25% compared to the average allocation scheme.

[0076] Step S600: The second-stage weight adjustment calculates the correlation between each effective modal feature and the target TCM gynecological syndrome through the attention mechanism, and performs a second fine-tuning on the first-stage adjustment weights to obtain the second-stage adjustment weight items.

[0077] It should be noted that the method for obtaining the second-stage adjustment weights includes: pre-screening candidate syndromes based on the first-stage adjustment weights, outputting a candidate syndrome set, and excluding syndromes with low probability; calculating the correlation score between each modality and the candidate syndrome set based on the single-modality feature vector set, obtaining the modality correlation score; generating weight fine-tuning coefficients, and then fine-tuning the first-stage adjustment weights based on the combination of the modality correlation score and the weight fine-tuning coefficients to obtain the second-stage adjustment weights.

[0078] Specifically, the essence of this step is to adapt the individual diagnostic differences of different patients to a lightweight attention mechanism under the hard constraints of the TCM gynecology knowledge graph. This addresses the pain point of the first stage's weights, which "only conform to general disease / staging rules and cannot match the prominence of individual characteristics." The clinical logic is as follows: different patients with the same disease and the same stage have vastly different core diagnostic criteria. For example, among patients with premenstrual polycystic ovary syndrome (PCOS), patient A may have prolactin levels that are more than three times the upper limit, making the test features highly identifiable. Patient B may have very typical symptoms of premenstrual breast tenderness and irritability, making the subjective symptoms highly identifiable. If the general weights of the first stage are used completely, such as 0.4 for test and 0.3 for symptoms, the core diagnostic features of some individuals will be diluted, and the diagnostic accuracy will decrease. The technical logic is as follows: unlike ordinary unconstrained pure data-driven attention mechanisms, the attention calculation in this step is completely anchored to the syndrome-feature association rules of the knowledge graph and a weight adjustment threshold is set, so it will not deviate from the basic logic of TCM diagnosis, while taking into account individual differences and clinical compliance.

[0079] Specifically, the adjustment weights of each effective mode output in the first stage are denoted as follows: Let i be the valid modality number. Missing modalities are not included in the calculation. The single-modality feature vector of each valid modality is denoted as Fi. The knowledge graph contains a pre-stored "standard feature library of candidate syndromes for the current disease". For example, the candidate syndromes for PCOS are three categories: liver qi stagnation, kidney deficiency and blood stasis, and phlegm-dampness obstruction. Each syndrome corresponds to a pre-labeled standard feature vector, which is generated by the weight combination of core related features. The pre-stored weight fine-tuning threshold in the knowledge graph is: the adjustment range of each modality does not exceed ±35% of the weight of the first stage to avoid deviating from the general diagnostic rules. Based on the weighted fusion features of the first stage, a lightweight classification is performed first to output the candidate syndrome set, excluding syndromes with extremely low probability, reducing the amount of subsequent attention calculation, and ensuring that the calculation logic conforms to the clinical doctor's thinking of "narrowing the syndrome range first and then making precise diagnosis". For each valid modality i and each candidate syndrome k, the correlation score is calculated:

[0080] ;

[0081] in The rule weight coefficient is fixed at 0.6 to ensure that knowledge rules have higher priority than data fitting. For rule-based association scoring, the number of matches between all features of this modality and the core features of the current candidate syndrome k is retrieved from the knowledge graph. For example, if three features of a certain modality match the core features of liver qi stagnation syndrome, then... The value is 0.9. If only one secondary feature is matched, then... It is 0.3. For data association, the cosine similarity between the current single-modal feature vector Fi and the standard feature vector of the candidate syndrome k is calculated, with a value range of [0,1]. Finally, the highest similarity among all candidate syndromes is taken. As the final correlation score for this mode, weight fine-tuning coefficients are then generated. For each valid mode i, the fine-tuning coefficients are calculated:

[0082] ;

[0083] in The average of the correlation scores for all valid modalities. To adjust the amplitude coefficient, a value of 0.35 is set, which perfectly matches the preset ±35% adjustment threshold. This means that the modality with the highest correlation will have its weight adjusted upwards by a maximum of 35%, and the modality with the lowest correlation will have its weight adjusted downwards by a maximum of 35%, without exceeding the preset rule boundaries. Finally, the personalized fusion weight is calculated and normalized. The initial personalized weight is obtained by multiplying the fine-tuning coefficient by the first-stage adjustment weight, i.e.:

[0084] ;

[0085] For all valid modes Normalization is performed to ensure that the sum of all modal weights is 1, ultimately yielding personalized fusion weights that can be directly used for fusion. This refers to the second stage of adjusting the weighting items.

[0086] Step S700: Based on the second-stage adjusted weight term as input data, input the pre-trained syndrome classification layer to obtain preliminary syndrome diagnosis results. Combine the completeness of the current input modality and the correlation between features and syndromes to calculate the confidence of the diagnosis results and obtain the result calculation term.

[0087] It is important to note that, such as Figures 4-5 As shown, the method for obtaining the result calculation item includes: calculating the modality completeness score based on the second-stage adjusted weight set and modality missing items to obtain the first calculation item; calculating the syndrome feature matching score based on the knowledge graph item, including core feature hit, secondary feature hit and conflict feature deduction, to obtain the second calculation item; and generating the final confidence score by fusing the first and second calculation items, while setting the confidence level and corresponding action to obtain the result calculation item.

[0088] Specifically, modality completeness is not simply calculated based on the proportion of modalities, but rather by weighting each modality according to its diagnostic weight within the current disease-stage, perfectly aligning with the diagnostic logic of Traditional Chinese Medicine gynecology.

[0089] ;

[0090] in This is the presence mask for the i-th modality, where valid data is 1 and missing data is 0. The initial weights for the i-th modality under the current disease-stage are pre-stored in the knowledge graph. The sum of the weights for the four modalities is 1. For example, the initial weights for the premenstrual phase of Polycystic Ovary Syndrome (PCOS) are 0.4 for diagnostic tests, 0.3 for the four diagnostic methods, 0.28 for symptoms, and 0.02 for imaging. If only the imaging modality is missing, then:

[0091]

[0092] If the test mode is missing, then:

[0093] ;

[0094] The clinical pattern of "PCOS having the greatest impact on syndrome differentiation based on the pre-examination modality" was fully matched. Then, the syndrome feature matching score was calculated. The feature matching score was calculated from three dimensions: core feature hit, secondary feature hit, and conflicting feature deduction, ensuring that the results conformed to the "core feature priority" rule of TCM syndrome differentiation. Specifically:

[0095] ;

[0096] in To determine the hit rate of the core features for diagnosing a syndrome, the ratio of the number of matched core features to the total number of core features for that syndrome is used, with a weighting of 70%. This aligns with the logic of TCM syndrome differentiation that "core features are sufficient to diagnose a syndrome." The hit rate of secondary features for diagnosing syndromes is calculated as the number of matched secondary features divided by the total number of secondary features in the effective modalities, with a weight of 30%. The score is calculated based on the number of conflicting features. 0.1 points are deducted for each conflicting feature, with a minimum deduction of 0 points. If there are ≥2 conflicting features, the matching degree is directly determined to be ≤0.7. Finally, the scores of the two classes are merged according to preset weights. Calibration items can also be added according to scenario requirements.

[0097] ;

[0098] in The completeness weight is set to 0.4 by default, meaning feature matching has higher priority than modality completeness. This is suitable for clinical scenarios where "some modalities are missing, but the core features are fully matched and still highly confident." The weight can be adjusted according to the scenario: it can be increased to 0.5 for tertiary hospitals and decreased to 0.3 for primary care / home-based scenarios. This is a calibration item for occult diseases, applicable only to asymptomatic gynecological conditions such as asymptomatic decreased ovarian reserve and early-stage endometriosis. If the core feature hit rate is ≥90%, a calibration score of 0.1-0.15 can be added. These diseases themselves have no obvious subjective symptoms, so this avoids misjudging as low confidence due to low modality integrity. The maximum score is 1 point. The output strategy is matched according to the scoring threshold, which can be flexibly adjusted according to the scenario. The general standards are shown in Table 3 below.

[0099] Table 3

[0100]

[0101] The supplementary feedback generation rule is: prioritize pushing the modality with the most deductions, if... If the deduction percentage is ≥60%, then the missing modality with the highest weight should be recommended for re-collection; if If the deduction rate is ≥60%, it is recommended to collect the currently missing modal that can supplement the core features. If the core features do not hit the pulse string, it is recommended to collect pulse diagnosis data. At the same time, inform the user that "collecting this modal can improve the confidence level". For example, collecting test data can improve the confidence level by 30%.

[0102] Step S800: Set a preset threshold, and based on the matching result of the result calculation item and the preset threshold, output the identification result to obtain the result output item, thereby realizing the intelligent identification of gynecological syndromes by integrating multimodal data.

[0103] It should be noted that the methods for obtaining the output items include: when the calculated items exceed the preset threshold, outputting the syndrome diagnosis results and the contribution explanation of each modality feature; when the confidence level does not exceed the preset threshold, pushing the feedback for supplementary sampling of missing modalities.

[0104] Specifically, when outputting high-confidence results, an explanation of the contribution of each modality feature is provided simultaneously, specifying the weight percentage of each core feature. For example, "Diagnosis of liver qi stagnation syndrome: 40% from elevated prolactin, 35% from wiry pulse and red tongue edges, and 25% from premenstrual breast tenderness." This avoids the problem of unverifiable black-box results. When pushing supplementary sampling feedback for low-confidence results, the modality with the highest deduction is prioritized, and the confidence gain after supplementary sampling is clearly stated, such as "It is recommended to supplement with AMH test, and the confidence can be increased by 32% after supplementary sampling," rather than the general "It is recommended to improve the examination." Supplementary sampling feedback can also be adapted to the scenario: in the case of a tertiary hospital, supplementary test / imaging examination is recommended, and in the case of home, supplementary pulse diagnosis and symptom follow-up are recommended. This output mechanism ensures the rigor of diagnosis and treatment while improving user acceptance.

[0105] A smart identification system for TCM gynecological syndromes based on multimodal physical sign fusion is disclosed. This system utilizes the aforementioned method for smart identification of TCM gynecological syndromes based on multimodal physical sign fusion, comprising: a graph construction module: pre-constructing a syndrome knowledge constraint graph, which includes initial weights for the diagnostic contributions of different gynecological diseases and different physiological stages, as well as core association features corresponding to each TCM gynecological syndrome, to obtain knowledge graph items; a data acquisition module: real-time acquisition of multimodal physical sign data of the object to be identified, to obtain real-time acquisition items, and standardized feature encoding of each modality data based on the real-time acquisition items to obtain a single-modality feature vector set; and a weight adjustment module: matching the single-modality feature vector set with the knowledge graph items to obtain missing modality data, to obtain missing modality items, and adjusting the weight based on the knowledge graph items. The atlas component employs a two-stage dynamic weight adjustment strategy to achieve multimodal feature fusion. The first stage weight adjustment is based on real-time acquisition data, resulting in the first-stage adjusted weight component. The second stage weight adjustment uses an attention mechanism to calculate the correlation between each effective modality feature and the target TCM gynecological syndrome, and performs a secondary fine-tuning of the first-stage adjusted weight to obtain the second-stage adjusted weight component. The result output module uses the second-stage adjusted weight component as input data, inputs it into a pre-trained syndrome classification layer to obtain preliminary syndrome diagnosis results, and calculates the confidence level of the diagnosis results by combining the completeness of the current input modality and the correlation between features and syndromes, resulting in the result calculation component. A preset threshold is set, and based on the matching result of the result calculation component and the preset threshold, the identification result is output, resulting in the result output component.

[0106] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended embodiments and their equivalents.

Claims

1. A method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion, comprising: A pre-constructed knowledge graph of syndromes is constructed. The graph includes the initial weights of the syndrome differentiation contributions of different gynecological diseases and different physiological stages, as well as the core association features corresponding to each TCM gynecological syndrome, to obtain knowledge graph items. Its characteristic is that it further includes: Multimodal vital signs data of the object to be identified are collected in real time to obtain real-time collection items. Based on the real-time collection items, the modal data are standardized and feature-encoded to obtain a single-modal feature vector set. By matching single-modal feature vector sets with knowledge graph items, missing modal data is obtained, and missing modal items are obtained. Based on knowledge graph items, a two-stage dynamic weight adjustment strategy is adopted to complete multimodal feature fusion. The first stage weight adjustment is based on real-time collected items to obtain the first stage adjusted weight items. The second-stage weight adjustment calculates the correlation between each effective modal feature and the target TCM gynecological syndrome through an attention mechanism, and performs a second fine-tuning on the first-stage adjustment weights to obtain the second-stage adjustment weight items. Using the second-stage adjusted weight term as input data, the pre-trained syndrome classification layer is input to obtain preliminary syndrome diagnosis results. The confidence of the diagnosis results is calculated by combining the completeness of the current input modality and the correlation between features and syndromes, resulting in a result calculation term. A preset threshold is set, and the identification result is output based on the matching result of the result calculation term and the preset threshold, thus achieving intelligent identification of gynecological syndromes by integrating multimodal data.

2. The method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion according to claim 1, characterized in that: The syndrome knowledge constraint graph has two types of built-in rules, and the methods for obtaining knowledge graph items include: The first type of rule in the syndrome knowledge constraint graph is the initial weight rule of disease stage modality. For each type of gynecological disease and each type of female physiological stage, different modal signs are pre-set with initial weights for their diagnostic contributions, resulting in initial diagnostic weight terms. The first type of rule in the syndrome knowledge constraint graph is the syndrome feature association priority rule, which pre-determines the priority ranking of the core association features corresponding to each TCM gynecological syndrome, and obtains the syndrome ranking item; The initial dialectical weight term and the syndrome ranking term are combined to obtain the knowledge graph term.

3. The method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion according to claim 1, characterized in that: The multimodal physical examination data includes subjective symptom time-series modal data, objective modal data of the four diagnostic methods, gynecological examination modal data, and gynecological imaging modal data. The methods for acquiring real-time data include: Subjective symptom temporal modal data includes menstrual history, marital and reproductive history, clinical subjective symptoms, and disease course information of the subject to be identified; objective modal data of the four diagnostic methods includes standardized collection of tongue, pulse, and facial images; gynecological laboratory modal data includes numerical test results of sex hormones, anti-Müllerian hormones, vaginal microecology, inflammatory factors, and coagulation function; and gynecological imaging modal data includes pelvic ultrasound and MRI imaging data.

4. The method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion according to claim 3, characterized in that: The method for obtaining the single-modal feature vector set includes: Based on the subjective symptom temporal modal data, a pre-trained TCM gynecology-specific word vector model was used to complete feature encoding. Based on the objective modal data of the four diagnostic methods, a lightweight convolutional neural network model was used to extract quantitative features to complete encoding. Based on the gynecological examination modal data, normalization processing was used to complete encoding. Based on the gynecological imaging modal data, a pre-trained medical imaging feature extraction model was used to extract lesion omics and blood perfusion features to complete encoding.

5. The method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion according to claim 1, characterized in that: The method for obtaining the missing modal term includes: The system obtains the number of categories in the single-modality feature vector set to get real-time category items. Based on the real-time category items, it performs category matching with knowledge graph items to determine whether there are missing categories in the single-modality feature vector set. When there are missing categories, it obtains the number of missing categories and the missing category feature data corresponding to the number of missing categories, and then obtains the modality missing items.

6. The method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion according to claim 1, characterized in that: The methods for obtaining the weight adjustment items in the first stage include: Based on the knowledge graph item, the initial weights corresponding to the single-modal feature vector set are retrieved from the knowledge graph item to obtain the weight item; Based on the obtained weight terms, the obtained weight terms are adaptively adjusted in the first round according to the missing modality terms. The initial weights of the missing modality terms are proportionally allocated to the corresponding features of the effective modalities of the same dimension to obtain the first-stage adjusted weight terms.

7. The method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion according to claim 1, characterized in that: The methods for obtaining the second-stage adjusted weight items include: Based on the first stage of adjusting the weights, candidate syndromes are pre-screened, a set of candidate syndromes is output, and low-probability syndromes are excluded. Based on the single-modal feature vector set, the correlation score between each modality and the candidate syndrome set is calculated to obtain the modality correlation score item; Generate weight fine-tuning coefficients, and then perform secondary fine-tuning on the first-stage adjusted weights based on the combination of modal correlation score items and weight fine-tuning coefficients to obtain the second-stage adjusted weight items.

8. The method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion according to claim 1, characterized in that: The method for obtaining the result calculation item includes: Based on the second-stage adjusted weight set and modal missing items, the modal completeness score is calculated to obtain the first calculation item; The matching score of syndrome features is calculated based on knowledge graph items, including core feature hit, secondary feature hit and conflict feature deduction, to obtain the second calculation item; The final confidence score is generated by fusing the first and second calculation items, and confidence levels and corresponding actions are set to obtain the result calculation items.

9. The intelligent identification method for TCM gynecological syndromes based on multimodal physical sign fusion according to claim 1, characterized in that: The methods for obtaining the output items include: When the calculated result exceeds the preset threshold, the syndrome diagnosis result and the contribution explanation of each modality feature are output. When the confidence level does not exceed the preset threshold, the feedback of supplementary sampling of missing modalities is pushed.

10. A TCM gynecological syndrome intelligent identification system based on multimodal physical sign fusion, characterized in that: The method for intelligent identification of TCM gynecological syndromes based on multimodal physical sign fusion, as described in any one of claims 1-9, includes: The graph construction module pre-constructs a syndrome knowledge constraint graph, which includes the initial weights of the syndrome differentiation contributions of different gynecological diseases and different physiological stages, as well as the core association features corresponding to each TCM gynecological syndrome, to obtain knowledge graph items; Data acquisition module: Real-time acquisition of multimodal vital sign data of the object to be identified, obtaining real-time acquisition items, and standardized feature encoding of each modal data based on the real-time acquisition items to obtain a single-modal feature vector set; Weight adjustment module: Based on the matching of single-modal feature vector set and knowledge graph items, missing modal data is obtained to obtain missing modal items. Based on knowledge graph items, a two-stage dynamic weight adjustment strategy is adopted to complete multimodal feature fusion. The first stage weight adjustment is based on real-time collected items to obtain the first stage adjusted weight items. The second stage weight adjustment calculates the correlation between each effective modal feature and the target TCM gynecological syndrome through the attention mechanism, and performs a second fine-tuning on the first stage adjusted weights to obtain the second stage adjusted weight items. The result output module uses the second-stage adjusted weight term as input data to input the pre-trained syndrome classification layer to obtain preliminary syndrome diagnosis results. It calculates the confidence of the diagnosis results by combining the completeness of the current input modality and the correlation between features and syndromes, and obtains the result calculation term. It sets a preset threshold and outputs the identification result based on the matching result of the result calculation term and the preset threshold, thus obtaining the result output term.

Images