Multi-modal data fusion-based threatened abortion risk prediction system and method

Abstract

The invention provides a threatened abortion risk prediction system and method based on multi-modal data fusion, and the system comprises a data collection and processing module which is used for collecting and preprocessing traditional Chinese medicine four-diagnosis data, fundus images and test data, and obtaining a preprocessed data set; the model training module is used for carrying out multi-modal feature extraction operation and multi-modal feature splicing operation on the basis of the preprocessed data set, and outputting a risk probability by combining feature importance screening and fusion training; wherein the multi-modal features comprise traditional Chinese medicine four-diagnosis features, fundus image features and test data feature codes; and the prediction analysis module is used for carrying out risk grade division based on the risk probability, outputting a traditional Chinese medicine syndrome type report and generating a treatment scheme in combination with the structured knowledge base. According to the method, macroscopic differentiation of traditional Chinese medicine and modern medical data are fused, prediction accuracy and clinical interpretability are improved, and intelligent decision support is provided for prevention and treatment of early pregnancy abortion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer application, and in particular to a preterm labor risk prediction method and system based on multi-modal data fusion. BACKGROUND

[0002] Miscarriage is a common complication in early pregnancy, as one of the important pregnancy diseases threatening the safety of mother and infant, it can exacerbate the pregnancy pressure of pregnant women, cause premature birth, congenital heart disease, developmental retardation and other diseases in newborns, and even lead to death of pregnant women and / or fetus. Threatened abortion refers to the phenomenon of vaginal bleeding in small amount after menstruation in women with positive urine and / or blood pregnancy test before 28 weeks of pregnancy, accompanied by closed cervix, uterus size consistent with the number of weeks of amenorrhea, normal fetal heart activity, no evidence of pregnancy products passing, no fetal or embryonic death.

[0003] Threatened abortion is diagnosed by vaginal bleeding with or without abdominal pain, so the current mainstream means to achieve prediction of threatened abortion is through blood test or ultrasound method. For example, reference 1 (Construction of machine learning tools to predict threatened miscarriage in the first trimester based on AEA, progesterone and β-hCG in China: a multicentre, observational, case-control study, BMC Pregnancy and Childbirth volume, 2022, 22, 697) discloses the construction of machine learning tools to predict threatened miscarriage in the first trimester based on AEA, progesterone and β-hCG in China: a multicentre, observational, case-control study, by establishing a machine learning model based on three hormones of endocannabinoid anandamide (AEA), progesterone (P4) and β-human chorionic gonadotropin (β-hCG) to predict the risk of threatened abortion.

[0004] For another example, reference 2 (Construction and verification of preterm labor nomogram risk prediction model for pregnant women, 2024, Military Nursing, 41(08), 52-55) applies Logistic regression analysis to the influencing factors of threatened abortion in pregnant women and constructs a nomogram risk prediction model. The fitting degree and prediction performance of the model are evaluated by using Hosmer-Lemeshow goodness-of-fit test, receiver operating characteristic curve, area under the curve and calibration curve, and Bootstrap self-sampling is used for internal validation of the model. The results show that the prediction model has good prediction effect and can provide reference basis for screening of high-risk groups of threatened abortion.

[0005] Traditional Chinese medicine has the advantage of overall conditioning in the prevention and treatment of threatened abortion, but the existing technology lacks the support of clinical data of traditional Chinese medicine diagnosis, and is mostly based on traditional single modal data such as blood test or ultrasound, which is difficult to fully reflect the pregnancy state. The multi-modal analysis of traditional Chinese medicine four diagnoses (face diagnosis, tongue diagnosis, pulse diagnosis, and interrogation) combined with modern medical imaging and test data can provide richer feature dimensions for risk prediction. SUMMARY

[0006] The purpose of the present application is to provide a threatened abortion risk prediction system and method based on multi-modal data fusion, which models the fusion of standardized collection data of traditional Chinese medicine and modern medical data, constructs a multi-branch neural network for high-precision multi-modal feature extraction, realizes quantitative evaluation of early pregnancy abortion risk and automatic identification of traditional Chinese medicine syndrome type, and provides a scientific basis for clinical intervention.

[0007] The threatened abortion risk prediction system based on multi-modal data fusion provided by the embodiment comprises: A data acquisition and processing module is used to acquire multi-modal data sets by collecting traditional Chinese medicine four diagnosis data, fundus images and test data, align the time stamps and bind them with unique patient identifiers, and perform preprocessing to obtain preprocessed data sets; wherein the preprocessed data sets include face images, tongue images, pulse diagnosis feature vectors, questionnaire text vectors, fundus images and test data; A model training module is used to perform multi-modal feature extraction, multi-modal feature splicing, and combined feature importance screening and fusion training based on the preprocessed data sets, and output risk probability; wherein the multi-modal features include traditional Chinese medicine four diagnosis features, fundus image features and test data feature encodings; A prediction analysis module is used to divide risk levels based on risk probability, quantify the contribution of each modal feature to generate high-risk indicator prompts, and output traditional Chinese medicine syndrome type reports, and generate treatment plans combined with structured knowledge bases.

[0008] In one embodiment, the multi-modal data set is obtained through a multi-source heterogeneous data acquisition system integrating standardized collection equipment of traditional Chinese medicine, fundus cameras and hospital test system interfaces.

[0009] Further preferably, the standardized collection equipment of traditional Chinese medicine includes a facial camera, a tongue diagnosis instrument and a pulse diagnosis instrument.

[0010] Further preferably, in the multi-modal data set, the format of traditional Chinese medicine four diagnosis data is one or more of image, signal and text; The format of the fundus image is JPEG; The format of the test data is CSV.

[0011] In one embodiment, the data acquisition and processing module further comprises: a data acquisition process for forming a standardized multi-modal data set through a manual input and automatic docking dual mode, compatible with the hospital inspection system data interface standard.

[0012] Further preferably, the unique patient identification binding includes a binding mechanism of the identity card number and the pregnancy number.

[0013] In one embodiment, the multi-modal feature extraction operation includes: inputting the preprocessed data set into a multi-branch neural network for multi-modal feature extraction; the multi-branch neural network includes: branch network one, branch network two and branch network three; The branch network one is used for inputting the preprocessed facial map, tongue map, pulse diagnosis feature vector and questionnaire text vector into an EfficientNet network embedded with a channel attention module, and outputting a layer to realize probability distribution calculation of four TCM syndromes through a Softmax function, and performing TCM four diagnostic semantic analysis to obtain TCM four diagnostic features; The branch network two is used for inputting the fundus image into a UNETR model constructed by a Transformer and a U-Net for fundus image segmentation, and combining ophthalmic disease data set transfer learning to obtain fundus image features; wherein the fundus image features include: retinal vascular density and / or vessel diameter asymmetry; The branch network three is used for converting the numerical sequence of the laboratory data into a time series feature vector through a bidirectional LSTM network of a text encoding module to obtain laboratory data feature encoding; wherein the laboratory data includes HCG and progesterone values.

[0014] In one embodiment, the four TCM syndromes include: kidney deficiency syndrome, kidney deficiency and blood heat syndrome, qi and blood deficiency syndrome, and spleen and kidney deficiency syndrome.

[0015] In one embodiment, the multi-modal feature splicing operation and the combined feature importance screening and fusion training include: inputting the TCM four diagnostic features, the fundus image features and the laboratory data feature encoding after splicing and dimensionality reduction into a gradient descent decision tree, and performing feature importance screening and fusion training with syndrome classification cross-entropy and risk regression mean square error as a joint loss function to output a risk probability.

[0016] In one embodiment, the risk level division based on the risk probability includes: dividing the risk probability into low-risk, medium-risk and high-risk risk levels, wherein the risk probability less than 30% is the low-risk level, the risk probability between 30%-70% is the medium-risk level, and the risk probability greater than 70% is the high-risk level.

[0017] In one embodiment, the quantification of the contribution of each modality feature to generate a high-risk indicator prompt includes: quantifying the feature contribution of the four diagnostic features of traditional Chinese medicine, fundus image features and laboratory data feature encoding through Shapley values ​​to generate a high-risk indicator prompt.

[0018] In one embodiment, the generation of a treatment plan in combination with a structured knowledge base includes: automatically matching and recommending personalized plans of Chinese medicine prescriptions, acupoint application or diet therapy through a structured knowledge base containing 4 types of Chinese medicine syndromes and 16 intervention plans, and generating a treatment plan including a risk factor radar chart and personal risk indicator prompts.

[0019] On the other hand, the present invention also provides a method for predicting the risk of threatened abortion based on multimodal data fusion, and the method for predicting the risk of threatened abortion based on multimodal data fusion is applied, comprising the following steps: Collecting data from the four diagnostic methods of Traditional Chinese Medicine, fundus images, and laboratory test data to obtain a multimodal dataset, aligning the multimodal dataset's timestamps with a unique patient identifier, and performing preprocessing to obtain a preprocessed dataset. The preprocessed dataset includes facial images, tongue images, pulse diagnosis feature vectors, questionnaire text vectors, fundus images, and laboratory test data. Based on the preprocessed data set, multimodal feature extraction and splicing operations are performed, and feature importance screening and fusion training are combined to output the risk probability; among them, the multimodal features include: the four diagnostic features of traditional Chinese medicine, fundus image features, and laboratory data feature encoding; Risk levels are divided based on risk probability, the contribution of each modal feature is quantified to generate high-risk indicator prompts, and a TCM syndrome report is output, which is then combined with a structured knowledge base to generate a treatment plan.

[0020] Compared with the prior art, the present invention has the following beneficial effects: (1) Advantages of multimodal fusion: The present invention combines the macroscopic syndrome differentiation of traditional Chinese medicine with the microscopic indicators of modern medicine to construct a multimodal fusion data set, which significantly improves the prediction results compared with single modality data and significantly improves the accuracy of predicted risk of threatened abortion.

[0021] (2) The multi-branch neural network constructed by the present invention focuses on strengthening the capture of key visual features such as sallow complexion and ecchymosis on the edge of the tongue, strengthening the acquisition of structural features in fundus images and improving generalization ability, and also strengthening the capture of dynamic changes in pregnancy indicators. It splices and fuses various modal features for training, realizes the two-way mapping of automatic identification of TCM syndrome types and risk quantification, and improves the clinical feasibility of the model.

[0022] (3) From data collection to report generation, the whole chain is automated, the clinical use threshold is reduced, the "syndrome-risk assessment-intervention suggestion" closed-loop decision logic is formed, and the linkage output of TCM syndrome type, risk probability and treatment scheme of threatened abortion risk is realized. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows.

[0024] Figure 1 The structure schematic diagram of the threatened abortion risk prediction system based on multi-modal data fusion provided by the present application is shown. Figure 2 The structure schematic diagram of the model training module provided in the embodiment of the present application is shown.

[0025] Figure 3 The structure schematic diagram of the branch network one in the embodiment of the present application is shown.

[0026] Figure 4 The flowchart of the threatened abortion risk prediction method based on multi-modal data fusion provided by the present application is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the present application more clear, the following will combine the drawings and the embodiments to further describe the present application in detail. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the protection scope of the present application.

[0028] The scheme of the embodiment of the present application is shown in Figure 1 A threatened abortion risk prediction system based on multi-modal data fusion, comprising: a data acquisition and processing module, a model training module and a prediction analysis module.

[0029] The data acquisition and processing module is used for acquiring multi-modal data sets by collecting TCM four diagnostic data, fundus image and test data, aligning the multi-modal data sets with time stamp and binding them with unique patient identification, and pre-processing to obtain pre-processed data sets; wherein the pre-processed data sets include: face map, tongue map, pulse diagnosis feature vector, questionnaire text vector, fundus image and test data.

[0030] In the embodiment, the desensitization data of 284 patients with threatened abortion in the hospital TCM gynecology department is collected, and the data content includes: patient ID number, contact number, age, last menstrual date, pregnancy mode, first hospitalization time after pregnancy, previous pregnancy times, adverse pregnancy times, left eye vision, right eye vision, left eye refraction, right eye refraction, left eye power, right eye power, left eye fundus image, right eye fundus image, tongue picture, face picture, left hand pulse L1 image, left hand pulse L2 image, left hand pulse L3 image, left hand pulse L4 image, right hand pulse R1 image, right hand pulse R2 image, right hand pulse R3 image, right hand pulse R4 image, data average diameter of gestational sac (mm), germ (mm), fetal heart (whether there is fetal heart), yolk sac (mm) (lri), whether there is intrauterine hemorrhage, RI (rri), progesterone P (noml / L), HCG (serum chorionic gonadotropin).

[0031] Then, a multi-source heterogeneous data acquisition system is obtained by deeply integrating the standardized collection equipment of traditional Chinese medicine, the fundus camera and the interface of the hospital test system. Specifically, the face camera and the tongue diagnosis instrument adopt ring light supplementing technology (light intensity is automatically calibrated to 300 lux-500 lux), which ensures the accurate collection of face color and tongue texture; the pulse diagnosis instrument is equipped with a single-channel pressure sensor to collect the pulse of the three parts of the body; the fundus camera is equipped with a 50° wide-angle lens, which automatically focuses on the macular area of the retina during shooting, and obtains color fundus images with a resolution of ≥300 DPI; the hospital test data is synchronized in real time through the HL7 interface, covering more than 10 pregnancy-related indicators such as HCG, progesterone and estradiol, and combining with the symptom questionnaire filled out by the patient (such as pregnancy times, vaginal bleeding status), to form a multi-modal data set. In the multi-modal data set, the format of the four diagnostic data of traditional Chinese medicine is one or more of image, signal and text; the format of the fundus image is JPEG; and the format of the test data is CSV.

[0032] All data are bound by a unique patient identifier (ID number + pregnancy number), which solves the traditional data island problem and ensures the accurate association of data from different sources, providing a consistent data basis for subsequent model training, constituting a unique data acquisition architecture and synchronization mechanism innovation point in the patent, and providing high-credibility raw data support for subsequent analysis.

[0033] In the embodiment, an adaptive preprocessing process, i.e., a hierarchical preprocessing pipeline, is designed according to the characteristics of multi-modal data. Specifically, based on image data, bilateral filtering algorithm and adaptive histogram equalization (CLAHE) are used for noise reduction and contrast enhancement of face and tongue, and U-Net model is used for segmentation of fundus image to extract retinal ROI region; For pulse diagnosis signal, 50Hz notch filter and wavelet transform are applied to extract characteristic parameters such as main wave amplitude and re-beat wave notch depth; TF-IDF vectorization is used for questionnaire text; KNN interpolation is used to fill in missing values and Z-score standardization is used for laboratory data; a TCM feature standardization algorithm is proposed, such as RGB to HSV color space conversion to quantify tongue color redness value (H∈[0,10]), moss texture complexity calculation based on LBP operator, and pulse diagnosis feature extraction rules based on pulse wave transit time (PTT) to evaluate blood vessel elasticity.

[0034] The model training module is used to extract multi-modal features, perform multi-modal feature concatenation, and combine feature importance screening and fusion training based on the preprocessed data set, and output risk probability; wherein the multi-modal features include TCM four diagnostic features, fundus image features, and laboratory data feature encoding.

[0035] As shown in the embodiment, Figure 2 The model training module adopts the architecture design of "three branch line parallel feature extraction + gradient fusion classification", realizing the deep fusion of TCM macroscopic syndrome differentiation and Western medicine microscopic indicators.

[0036] First, as shown in the embodiment, Figure 3 The branch network one is based on an improved EfficientNet-B4 network, which embeds a channel attention module (SE-Net) in the convolutional layer, focuses on strengthening the capture of key visual features such as facial yellowish and tongue edge ecchymosis, and realizes the probability distribution of four TCM syndromes (kidney deficiency, kidney deficiency and blood heat, qi and blood deficiency, and spleen and kidney deficiency) through the Softmax function in the output layer, and outputs a deep feature vector.

[0037] Next, the branch network two uses the UNETR model (Transformer+U-Net architecture) to segment the fundus ROI image, combines the ophthalmic disease data set transfer learning, extracts structural features such as blood vessel density and pipe diameter asymmetry, and loads the ophthalmic pre-training weight to improve the generalization ability.

[0038] The third branch network uses the bidirectional LSTM network of the Text Encoder module to model the time sequence of the laboratory index sequence, capture dynamic features such as HCG doubling rate and progesterone fluctuation trend, and specifically convert the HCG and progesterone numerical sequence into time sequence feature vectors that capture dynamic trends, to obtain laboratory data feature encoding.

[0039] Finally, the modal features containing TCM four diagnostic features, fundus image features, and laboratory data feature encoding are concatenated and dimensionally reduced, and input into the gradient descent decision tree (GBDT), and the syndrome classification cross-entropy and risk regression mean square error are used as the joint loss function for training, and the risk probability is output, and the feature contribution degree is quantified through the integrated Shapley value calculation engine, and the formula is as follows: , , , in, and They are the classification cross entropy loss function and the risk regression mean square error loss function respectively. N is the sample size, C is the number of categories, It is a sample Belong to category c The true label, is the model prediction sample Belong to category c The probability of is a balancing parameter that controls the relative weights of the two loss functions. This module combines EfficientNetB4 with the attention mechanism for TCM syndrome classification and designs a UNETR+LSTM cross-modal fusion path to achieve interpretable risk prediction. This constitutes the core innovation of multimodal model architecture and joint training technology to achieve risk prediction.

[0040] The predictive analysis module is used to classify risk levels based on risk probability, quantify the contribution of each modal feature to generate high-risk indicator prompts, and output a TCM syndrome report, and then generate a treatment plan in combination with a structured knowledge base.

[0041] In the embodiment, the module is based on a trained multi-branch neural network and divides risk levels into low, medium, and high risk levels according to the GBDT output probability. Among them, the risk probability less than 30% is low risk, the risk probability between 30% and 70% is medium risk, and the risk probability greater than 70% is high risk. High-risk indicators such as age and HCG abnormalities are annotated by Shapley values, and a TCM syndrome type report is output. Then, through the built-in structured knowledge base containing 4 syndrome types and 16 intervention plans, it can automatically match and recommend personalized plans such as Chinese medicine prescriptions (such as Baoyin Decoction), acupoint application (such as Shenshu acupoints), and dietary therapy (such as fetal protection carp soup). It also generates plans including risk factor radar charts and personal risk indicator prompts, forming a closed-loop decision-making logic of "syndrome differentiation-risk assessment-intervention prompts", realizing the linked output of TCM syndrome types, risk probabilities, and treatment plans.

[0042] like Figure 4 As shown, the embodiment also provides a method for predicting the risk of threatened abortion based on multimodal data fusion, comprising the following steps: S1, obtain a multi-modal data set by collecting traditional Chinese medicine four diagnostic data, fundus image data and laboratory data, align the multi-modal data set with a time stamp and bind it with a unique patient identifier, and perform preprocessing to obtain a preprocessed data set; wherein the preprocessed data set comprises: a face map, a tongue map, a pulse diagnosis feature vector, a questionnaire text vector, fundus image data and laboratory data; S2, perform multi-modal feature extraction, multi-modal feature splicing, and combined feature importance screening and fusion training based on the preprocessed data set, and output a risk probability; wherein the multi-modal features include: traditional Chinese medicine four diagnostic features, fundus image features and laboratory data feature encodings; S3, based on the risk probability, divide the risk level, quantify the contribution of each modality feature to generate a high-risk index prompt, and output a traditional Chinese medicine syndrome type report, and then generate a treatment plan combined with a structured knowledge base.

[0043] It should be noted that the above embodiment provides a pre-eclampsia risk prediction system and method based on multi-modal data fusion. When predicting the risk of pre-eclampsia, the above-mentioned functional modules are divided, and the above-mentioned functions can be completed by different functional modules according to the needs, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the pre-eclampsia risk prediction system and method based on multi-modal data fusion provided by the above embodiment belong to the same concept, and the specific implementation process is described in detail in the pre-eclampsia risk prediction system based on multi-modal data fusion. Here, it is not repeated.

[0044] In summary, the present application constructs a multi-modal fusion data set by combining traditional Chinese medicine macroscopic syndrome differentiation and modern medicine microscopic indicators. Compared with the results of single modality data prediction, the prediction accuracy of pre-eclampsia risk prediction is improved by more than 30%. In the model design, a variety of segmentation and classification algorithms are selected, including but not limited to EfficientnetB4, UNETR, TXT Encoder and GBDT. The key visual features such as sallow face and tongue edge ecchymosis are captured, the structural features in the fundus image are acquired and the generalization ability is improved, and the dynamic change rule of pregnancy indicators is captured. The various modal features are spliced and fused for training, realizing the bidirectional mapping of automatic identification of traditional Chinese medicine syndrome type and risk quantification, and improving the clinical feasibility of the model. And from data acquisition to report generation, the whole chain is automated, the clinical use threshold is reduced, a "syndrome differentiation-risk assessment-intervention prompt" closed-loop decision logic is formed, and the linkage output of traditional Chinese medicine syndrome type, risk probability and treatment plan of pre-eclampsia risk is realized.

[0045] The above detailed description of the specific embodiments of the present application has described the technical solutions and beneficial effects of the present application, and it should be understood that the above description is only the most preferred embodiment of the present application and is not intended to limit the present application. Any modifications, supplements and equivalent replacements made within the principle range of the present application shall be included in the protection range of the present application.

Claims

1. Data acquisition and processing module, used to collect TCM four diagnostic data, fundus images and laboratory test data to obtain a multimodal data set, align the timestamp of the multimodal data set with the unique patient identifier, and perform preprocessing to obtain a preprocessed data set; wherein, The preprocessed dataset includes: facial images, tongue images, pulse diagnosis feature vectors, questionnaire text vectors, fundus images, and laboratory data; The model training module is used to extract and concatenate multimodal features based on the preprocessed dataset, and then combine feature importance screening and fusion training to output risk probabilities. Multimodal features include: the four diagnostic features of Traditional Chinese Medicine, fundus image features, and laboratory data feature encoding. The predictive analysis module is used to classify risk levels based on risk probability, quantify the contribution of each modal feature to generate high-risk indicator prompts, and output a TCM syndrome report, and then generate a treatment plan in combination with a structured knowledge base.

2. The threatened abortion risk prediction system according to claim 1, characterized in that: The multimodal dataset is obtained through a multi-source heterogeneous data acquisition system that integrates traditional Chinese medicine standardized acquisition equipment, fundus cameras, and hospital testing system interfaces.

3. The threatened abortion risk prediction system according to claim 2, characterized in that: The data acquisition and processing module also includes: a dual mode of manual entry and automatic docking, compatible with the data interface standard of the hospital testing system, to form a standardized multimodal data set acquisition process.

4. The threatened abortion risk prediction system according to claim 1, wherein: The multimodal feature extraction operation includes: inputting the preprocessed data set into a multi-branch neural network for multimodal feature extraction; the multi-branch neural network includes: branch network 1, branch network 2 and branch network 3; The branch network 1 is used to input the preprocessed facial image, tongue image, pulse diagnosis feature vector and questionnaire text vector into the EfficientNet network embedded by the channel attention module. The output layer uses the Softmax function to calculate the probability distribution of the four TCM syndrome types and perform semantic analysis of the four TCM diagnoses to obtain the characteristics of the four TCM diagnoses. The second branch network is used to input the fundus image into the UNETR model constructed by Transformer and U-Net to perform fundus image segmentation, and combine it with the ophthalmic disease dataset for transfer learning to obtain fundus image features; the fundus image features include: retinal vessel density and / or vessel diameter asymmetry; The branch network three is used to convert the numerical sequence of the test data into a time series feature vector through the bidirectional LSTM network of the text encoding module to obtain the feature code of the test data; wherein the test data includes HCG and progesterone values.

5. The threatened abortion risk prediction system according to claim 4, characterized in that: The four TCM syndromes include: kidney deficiency syndrome, kidney deficiency and blood heat syndrome, qi and blood deficiency syndrome, and spleen and kidney deficiency syndrome.

6. The threatened abortion risk prediction system according to claim 4, characterized in that: The multimodal feature splicing operation and the combined feature importance screening and fusion training include: encoding and splicing the four diagnostic features of traditional Chinese medicine, fundus image features and laboratory data features, reducing the dimensionality and inputting them into the gradient descent decision tree, using the syndrome classification cross entropy and the risk regression mean square error as the joint loss function to perform feature importance screening and fusion training, and outputting the risk probability.

7. The threatened abortion risk prediction system according to claim 6, characterized in that: The risk level classification based on risk probability includes: classification into low risk, medium risk and high risk levels based on risk probability, wherein a risk probability less than 30% is a low risk level, a risk probability between 30%-70% is a medium risk level, and a risk probability greater than 70% is a high risk level.

8. The threatened abortion risk prediction system according to claim 4, characterized in that: The quantification of the contribution of each modality feature to generate a high-risk indicator prompt includes: quantifying the feature contribution of the four diagnostic features of traditional Chinese medicine, fundus image features and laboratory data feature coding through Shapley values ​​to generate a high-risk indicator prompt.

9. The threatened abortion risk prediction system according to claim 5, characterized in that: The generation of treatment plans in combination with a structured knowledge base includes: automatically matching and recommending personalized plans of Chinese medicine prescriptions, acupoint application or diet therapy through a structured knowledge base containing 4 types of Chinese medicine syndromes and 16 intervention plans, and generating a treatment plan including a risk factor radar chart and personal risk indicator prompts.

10. A method for predicting the risk of threatened abortion based on multimodal data fusion, characterized in that: The threatened abortion risk prediction system based on multimodal data fusion according to any one of claims 1 to 9 comprises the following steps: Collecting data from the four diagnostic methods of Traditional Chinese Medicine, fundus images, and laboratory test data to obtain a multimodal dataset, aligning the multimodal dataset's timestamps with a unique patient identifier, and performing preprocessing to obtain a preprocessed dataset. The preprocessed dataset includes facial images, tongue images, pulse diagnosis feature vectors, questionnaire text vectors, fundus images, and laboratory test data. Based on the preprocessed data set, multimodal feature extraction and splicing operations are performed, and feature importance screening and fusion training are combined to output the risk probability; among them, the multimodal features include: the four diagnostic features of traditional Chinese medicine, fundus image features, and laboratory data feature encoding; Risk levels are divided based on risk probability, the contribution of each modal feature is quantified to generate high-risk indicator prompts, and a TCM syndrome report is output, which is then combined with a structured knowledge base to generate a treatment plan.

Images