A multimodal risk assessment and decision-making support method for contraceptive safety monitoring

Abstract

The present invention provides a multimodal risk assessment and decision-making support method for contraceptive safety monitoring, which belongs to the field of deep learning-based risk assessment technology. First, multimodal medical data of contraceptive users is collected to construct a standardized risk assessment dataset. Subsequently, a multi-channel risk prediction model is constructed to perform deep feature modeling on the multimodal data separately, and fuse them to generate a unified representation for synchronously predicting multiple risk outcomes. Next, a risk mechanism modeling and path inference module is designed to quantify the independent contribution of each modality to the prediction results, and to construct an unsupervised risk propagation path based on a keyword mechanism to reveal the potential correlation mechanism between input features and risk outcomes. Finally, a comprehensive risk assessment result is generated. Based on the modal contribution vector and the optimal risk propagation path information, decision-making support prompts are provided to doctors. The present invention improves the efficiency of identifying potential risks of contraceptive use, the accuracy of judgment, and the clinical usability.

Description

Technical Field

[0001] The present invention belongs to the technical field of risk assessment based on deep learning, and in particular relates to a multimodal risk assessment and decision-making support method for contraceptive safety monitoring. Background Art

[0002] The scientific use of contraceptives is a vital component of modern reproductive health systems and a key public health intervention for protecting female fertility and preventing unwanted pregnancies and their complications. Providing scientific, accessible contraceptive services to women throughout their 30-year reproductive lifespan effectively ensures birth spacing and significantly reduces induced abortion rates and the associated risks of endometrial damage, infection, and infertility.

[0003] The widespread use of contraceptives covers women of different age groups, physical types and menstrual cycle status, and has the characteristics of long use cycle, high compliance requirements and significant individual differences.

[0004] However, in actual use, the use of contraceptives is often accompanied by complex physiological responses, and there is a certain proportion of individual risk of adverse reactions or complications. For example, some users may experience liver and kidney dysfunction, cardiovascular reactions, endocrine disorders, or hormone metabolism disorders. These risks are not only related to the use of the drug itself, but also influenced by factors such as the individual's basic background, physiological state, drug metabolism capacity, and previous medical history. They are highly heterogeneous and often exhibit latent and lagging effects under chronic accumulation or cross-effects, making them difficult to identify in a timely manner at an early stage.

[0005] At present, in order to achieve the evaluation and monitoring of potential risks during the use of contraceptives, common methods mainly include the following three categories:

[0006] Qualitative monitoring methods based on questionnaires and manual assessments: This method collects information on subjective health changes before and after use (such as menstrual cycles, weight fluctuations, mood changes, etc.) and combines it with the doctor's experience to make qualitative judgments. It has the advantages of simple operation and low cost, and is suitable for grassroots follow-up and routine screening. However, this method relies heavily on individual subjective statements and the experience of medical staff. It lacks objectivity and structured expression, making it difficult to carry out system modeling and trend prediction, especially the ability to identify early organ dysfunction.

[0007] Risk scoring methods based on single-modal medical data: These methods use medical test results (such as liver function indicators and biochemical tests) as a single feature input and utilize models such as logistic regression and linear discriminant to establish a risk scoring mechanism. This method improves the quantification and interpretability of the assessment to a certain extent, but because it relies solely on a single modality (such as structured indicators or images), it cannot effectively integrate information dimensions such as clinical text, behavioral habits, and time series. The model's expressive power is limited, making it difficult to adapt to the risk assessment needs in complex individual contexts.

[0008] End-to-end classification models based on deep learning: Some technologies attempt to introduce deep neural network structures such as CNN and LSTM to perform end-to-end learning on clinical data to achieve automated prediction of adverse reactions. Although such models have made some progress in accuracy, they generally suffer from insufficient interpretability and are unable to provide a clear understanding of mechanisms or modal contribution analysis. In addition, such methods are often limited to modeling structured or time-series data and lack the systematic integration of multimodal medical information (such as images and text). As a result, there is still considerable room for improvement in the model's generalization ability and clinical credibility. Summary of the Invention

[0009] To address the above issues, the present invention proposes a multimodal risk assessment and decision-making support method for contraceptive safety monitoring, which includes the following steps:

[0010] S1, collects multimodal medical data related to contraceptive use, including structured demographic information, clinical diagnosis and treatment texts, and medication time series data;

[0011] S2, inputs the collected data in S1 into the trained contraceptive multi-risk prediction model. The model includes three independent modal channels, which respectively encode and model basic demographic information, clinical text, and medication behavior sequences, and fuse them to generate a unified representation for predicting and outputting multiple key risk indicators, including the probability of organ dysfunction, adverse reaction type, and adverse reaction severity level;

[0012] S3, based on the contraceptive pill risk prediction model trained in S2, performs independent prediction tests on each modal input feature in turn, and calculates the joint prediction results of the three modalities to obtain the modal contribution vector;

[0013] Based on structured demographic information, clinical diagnosis and treatment text information, and medication time series information, key medical information contained in each information is extracted to construct the optimal risk transmission path;

[0014] S4 generates a comprehensive risk assessment result based on the prediction results of S2 and the set risk level judgment rules; at the same time, it provides doctors with auxiliary decision-making prompts based on the modal contribution vector and the optimal risk propagation path information.

[0015] Preferably, the multimodal medical data specifically includes:

[0016] Structured demographic information: including basic medical background variables such as age, gender, reproductive history, past medical history, and family medical history of contraceptive users; and organizing and splicing this information into a basic demographic information vector ;

[0017] Clinical diagnosis and treatment text information: including unstructured text such as doctor's consultation notes, diagnosis results, prescription information, and adverse reaction descriptions ;

[0018] Medication time series information: records the medication use records of contraceptive users in each time period during the entire use cycle, including: drug name and type, manufacturer number, production batch number, medication dosage, medication frequency, and missed or missed doses; thus obtaining the medication time series data of contraceptive users ,and ;in, Indicates in Medication records for a period of time, and .

[0019] Preferably, the probability of organ dysfunction includes expert scoring of the probability of liver and kidney damage, cardiovascular dysfunction, reproductive system abnormality, and nervous system dysfunction to obtain an organ dysfunction vector label;

[0020] The adverse reaction types are coded according to common clinical types, including gastrointestinal system reactions (including abdominal pain, diarrhea, gastrointestinal reactions, etc.), endocrine disorders (including menstrual problems, breast tenderness, weight changes, etc.), skin and discoloration reactions (including acne, facial pigmentation, chloasma, etc.), vascular system abnormalities (including hypertension, venous thrombosis, lower limb edema, etc.), nervous system problems (including headache, insomnia, dizziness, etc.), and other reactions, to obtain adverse reaction coding vector labels;

[0021] The severity level of the adverse reaction is labeled from 1 to 5 according to the WHO international standard, and the higher the score, the more severe the adverse reaction.

[0022] Preferably, the contraceptive multi-risk prediction model includes a basic risk modeling channel, a clinical event identification channel, and a behavioral compliance modeling channel;

[0023] The basic risk modeling channel is based on the basic population information vector As input, we get the risk background characteristics ;

[0024] The clinical event identification channel is based on clinical diagnosis and treatment text data As input, the output features of the clinical event recognition channel are obtained ;

[0025] The behavioral compliance modeling channel is based on drug time series data As input, the output features of the generated behavior compliance modeling channel are obtained ;

[0026] Finally, the output features will be 、 、 、 Forming joint features through channel splicing ; Then the multi-head attention mechanism is introduced to achieve the saliency weighting between modalities and obtain the fusion representation , Input two multi-layer perceptron modules in sequence for nonlinear mapping and dimension compression, and generate the final fusion risk representation through the ReLU activation function ;

[0027] Final fusion risk representation Input three sets of prediction heads and output the probability of organ dysfunction, adverse reaction type and adverse reaction severity level.

[0028] Preferably, the basic risk modeling channel first performs nonlinear feature expansion through two serially connected KAN function encoding layers, wherein each KAN function encoding layer is composed of a basic function layer, a projection mapping layer based on a fully connected layer, and a Dropout layer, for accurately fitting and combining different types of structural variables;

[0029] Then the fitting is strengthened again through the KAN function encoding layer, and then the nonlinear expression is enhanced through the SiLU activation function to obtain the primary structural risk vector , and use the ResNet residual structure to convert the original input and Splicing is performed to obtain the output features of the first set of double-layer KAN-SiLU-Resnet combination ; Then, The high-order feature combination is performed again through the double-layer KAN-SiLU-Resnet combination, and the output feature is obtained ; Finally, through the fully connected layer Map to a unified fusion dimension and obtain risk background characteristics .

[0030] Preferably, the clinical event recognition channel first converts the original text sequence into is the word vector representation , then continuously through 5 chained Transformer encoding layers to perform contextual semantic modeling, capturing the potential event relationship and diagnostic semantic structure in the medical text; then through the fully connected layer to compress the data dimension and generate a unified dimensional data representation to obtain the output features of the clinical event recognition channel .

[0031] Preferably, the behavior compliance modeling channel first uses the GRU layer to capture the dynamic changes of short-term medication status and outputs the behavior embedding ; Then the ESN layer is introduced to further model long-term behavioral dependencies and obtain long-term behavioral state representation , and generate the initial behavioral risk features through the Tanh activation function ; Repeatedly input the GRU-ESN-Tanh combination structure for deep dynamic modeling and output high-order behavioral features Finally, the feature dimensions are unified through the fully connected layer to generate the output features of the behavior compliance modeling channel .

[0032] Preferably, the specific process of obtaining the modal contribution vector in S3 is:

[0033] Based on the trained contraceptive risk prediction model, each modal input feature is tested independently in turn. During the test, other pathways are frozen to obtain independent response outputs of the three modalities. 、 、 ;

[0034] Based on combining the above modal independent outputs with the complete model to predict the output , calculate the modal contribution ; Specifically, calculate the mode-independent output (It is 、 、 One of the values) is combined with the full model to predict the output The L2 norm of the difference between The L2 norm is normalized, and the inverse of the calculated result is taken as the contribution value of the mode. The calculation formula is:

[0035] ;

[0036] in, represents the modal contribution value, Indicates calculation of L2 norm; for 、 、 One of the values.

[0037] Preferably, the specific process of constructing the optimal risk propagation path in S3 is:

[0038] S31, based on structured demographic information , clinical diagnosis and treatment text information and medication time series information , using the pre-trained ClinicalBERT medical language model to extract key medical information contained in each message and generate text keywords;

[0039] Based on all Keywords get keyword set ,in Indicates the keyword text, and ;

[0040] S32, each keyword is regarded as a node, and the keyword text Obtain the corresponding semantic features through the Word2Vec embedding layer , and use it as the feature of each keyword node;

[0041] S33, calculate the probability of organ dysfunction The average value of the organ risk intensity coefficient ; Prediction results for adverse reaction categories , take the probability value corresponding to the most likely adverse reaction category as the adverse reaction risk coefficient ; In addition, the results are classified according to the severity level The probability values ​​of severity levels 1 to 5 are weighted and summed to obtain the severity level risk coefficient. ;

[0042] For any pair of keyword nodes in the graph, the embedding feature vectors of the two nodes are and Define edge direction weights as a basis ; Calculate the direction weights of all edges; and Respectively represent Hedi keyword node features, and ;

[0043] S34, the length of the preset risk transmission path is , starting from any keyword node in the mechanism diagram, Under the constraint of , a greedy search algorithm is used to select the outgoing edge with the largest edge weight among the currently reachable nodes, gradually construct the path, and accumulate the path edge weights; finally, the path with the largest sum of path edge weights in the entire graph is extracted to form the optimal path for risk propagation. .

[0044] Preferably, the probability of organ dysfunction in S4 is based on , adverse reaction category , severity classification results A four-level comprehensive risk assessment rule has been formulated, including:

[0045] Level I low risk: probability of organ dysfunction , severity level , and the type of adverse reaction Common mild type, meaning there is basically no systemic impact and only routine monitoring is required;

[0046] Level II medium risk: probability of organ dysfunction , severity level ; or adverse reaction type Involvement of the digestive system or endocrine system indicates that vigilance and regular follow-up are recommended;

[0047] Level III high risk: probability of organ dysfunction ; or severity level ; or adverse reaction type For central nervous system reactions, active intervention and follow-up observation are required;

[0048] Level IV Very high risk: severe level , and is accompanied by a high probability of organ dysfunction , and high-risk adverse reaction types require immediate intensive assessment and monitoring measures.

[0049] Compared with the prior art, the invention has the following innovative features and beneficial effects:

[0050] (1) Design of a multimodal and multi-risk prediction model: Aiming at the task of monitoring the safety of contraceptive pills, a multimodal deep prediction model is constructed that integrates structured demographic information, clinical text, and medication time series behavior. Through three-channel independent modeling and multi-risk joint prediction, the system realizes the collaborative identification of multiple types of potential physiological risks, significantly improving the comprehensiveness and prediction accuracy of individualized risk perception.

[0051] (2) Mechanism modeling and risk path reasoning based on task feedback guidance: A risk mechanism reasoning method combining modal response evaluation and keyword-level path modeling was designed. Through unsupervised mechanism graph construction, risk response-guided edge weight calculation, and greedy path search algorithm, the input feature propagation chain behind high-risk prediction results was systematically mined, enhancing the causal interpretation ability of the model output and the traceability of risk evolution.

[0052] (3) Comprehensive risk assessment and auxiliary prompt mechanism integrating prediction results and mechanism information: A comprehensive risk level determination strategy combining multiple risk prediction results, modal contribution vectors and risk propagation paths is proposed, and a four-level risk output rule is set to support structured clinical risk classification; at the same time, through dominant modality identification and mechanism chain description, the system can generate auxiliary feedback text with explainability and prompts, thereby enhancing the clinical practicality of the model output and the efficiency of doctor response. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A logic flow chart for the overall implementation of the present invention.

[0054] Figure 2 This is a network structure diagram of the multi-risk prediction model for contraceptives of the present invention.

[0055] Figure 3 These are the comparison results of the organ dysfunction probability prediction experiment in the embodiments of the present invention.

[0056] Figure 4 These are comparative results of the adverse reaction classification prediction experiment in the examples of the present invention.

[0057] Figure 5 These are the comparative results of the adverse reaction severity classification prediction experiment in the embodiments of the present invention.

[0058] Figure 6 This is a heat map of the contribution of three types of modes in ten test samples in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] This invention proposes a multimodal risk assessment and decision-making support method for contraceptive safety monitoring. The overall technical process is as follows: Figure 1 As shown:

[0060] S1. Contraceptive multimodal data collection and risk assessment dataset construction: Collect multimodal medical data related to contraceptive use, including structured demographic information, clinical diagnosis and treatment texts, and medication time series data; label the data with labels such as organ dysfunction probability, adverse reaction type, and adverse reaction severity level; and construct a contraceptive risk assessment dataset for model training and risk modeling.

[0061] S2, Constructing a multi-risk prediction model for contraceptives: Based on the dataset constructed in S1, a risk prediction model with a multimodal input structure was constructed. The model includes three independent modal channels, which respectively encode and model basic demographic information, clinical text, and medication behavior sequences, and fuse them to generate a unified representation for predicting multiple key risk indicators, including the probability of organ dysfunction, the type of adverse reaction, and the severity level.

[0062] S3, Risk Mechanism Modeling and Pathway Inference Module Design: Based on the original input provided by S1 and the multiple risk prediction results output by S2, a path reasoning submodule, including a modal response analysis submodule and a mechanism path reasoning submodule, is constructed. The modal response analysis submodule is used to evaluate the independent contribution of each modality to the prediction results, forming a modal contribution vector. The path reasoning submodule extracts key medical information contained in structured demographic information, clinical diagnosis and treatment text information, and medication time series information to reveal the potential mechanistic association between input features and risk outcomes.

[0063] S4, comprehensive risk assessment and decision-making assistance method: Based on the three types of prediction results of S2 and the set risk level judgment rules, a comprehensive risk assessment result is generated; at the same time, based on the modal contribution vector and propagation path information, auxiliary decision-making prompts are provided to doctors.

[0064] The present invention will be further described below with reference to specific embodiments.

[0065] 1. Contraceptive Multimodal Data Collection and Risk Assessment Dataset

[0066] To comprehensively characterize the potential risks of contraceptive use, we first need to systematically collect multimodal input data and standardize the training dataset. This involves the following steps:

[0067] 1. Contraceptive pill multimodal input data collection: Collect multimodal medical data related to contraceptive pill use, specifically including the following three types of modal information:

[0068] (1) Structured demographic information: including basic medical background variables such as age, gender, reproductive history, past medical history, and family medical history of contraceptive users; and arranging and splicing this information into a basic demographic information vector ;

[0069] (2) Clinical diagnosis and treatment text information: including unstructured text of doctor's consultation records, diagnosis results, prescription information and adverse reaction descriptions ;

[0070] (3) Medication time series information: Record the medication records of contraceptive users in each time period during the entire use cycle, including: drug name and type, manufacturer number, production batch number, medication dosage, medication frequency, and missed or missed doses; thus obtaining the medication time series data of contraceptive users ,and ;in, Indicates in Medication records for a period of time, and ;

[0071] The above data together constitute the multimodal input data of contraceptive pills ;

[0072] 2. Multi-risk labeling: Based on medical knowledge and clinical feedback, professional physicians are organized to collect multimodal data. Carry out risk labeling, the label content includes:

[0073] (1) Probability labeling of organ dysfunction, including expert scoring of the probability of liver and kidney damage, cardiovascular dysfunction, reproductive system abnormality, and nervous system dysfunction, and finally obtaining the organ dysfunction vector label ;

[0074] (2) Adverse reaction type labeling: Coding is performed according to common clinical types, including gastrointestinal system reactions (including abdominal pain, diarrhea, gastrointestinal reactions, etc.), endocrine disorders (including menstrual problems, breast pain, weight changes, etc.), skin and discoloration reactions (including acne, facial pigmentation, chloasma, etc.), vascular system abnormalities (including hypertension, venous thrombosis, lower limb edema, etc.), nervous system problems (including headache, insomnia, dizziness, etc.) and other reactions, to obtain adverse reaction coding vector labels. ;

[0075] (3) Labeling of the severity of adverse reactions: Label the severity of adverse reactions according to the WHO international standard from 1 to 5 (the higher the score, the more severe the adverse reaction). ;

[0076] Based on the above labeling method, the multi-risk output label of contraceptive pills is ;

[0077] 3. Risk assessment dataset construction: collect multimodal data of contraceptive pills As input data, annotated multiple risk labels As output data, a set of risk assessment datasets was obtained. Based on this method, extensive data collection and dataset production were carried out for different contraceptive users, and finally a complete contraceptive risk assessment dataset was obtained. .

[0078] 2. Construction of a multi-risk prediction model for contraceptive pills

[0079] In order to accurately predict the potential risks during the use of contraceptives, the present invention constructs a multi-risk prediction model for contraceptives. As input, including basic population information vector , clinical diagnosis and treatment texts and medication time series data The prediction output includes multiple key risk indicators such as the probability of organ dysfunction, the type of adverse reaction and its severity. Specifically, the model adopts a multimodal branch structure consisting of three independent input channels, and performs modal fusion and risk indicator prediction in sequence. The network structure of the contraceptive multi-risk prediction model is as follows: Figure 2 As shown;

[0080] 1. Basic risk modeling channel: This channel is used to model the association between individual static background attributes and long-term risks, using basic population information vectors As input, it first performs nonlinear feature expansion through two serially connected KAN (Kolmogorov-Arnold Networks) function encoding layers. Each KAN function encoding layer consists of a basic function layer (including polynomial basis functions, B-spline functions, and Fourier basis functions), a projection mapping layer based on a fully connected layer, and a Dropout layer, which is used to accurately fit and combine different types of structural variables.

[0081] Then the fitting is strengthened again through the KAN function encoding layer, and then the nonlinear expression is enhanced through the SiLU activation function to obtain the primary structural risk vector , and use the ResNet residual structure to convert the original input and Splicing is performed to obtain the output features of the first set of double-layer KAN-SiLU-Resnet combination ; Then, The high-order feature combination is performed again through the double-layer KAN-SiLU-Resnet combination, and the output feature is obtained ; Finally, through the fully connected layer Map to a unified fusion dimension and obtain risk background characteristics , which is used as the output feature of the basic risk modeling channel;

[0082] 2. Clinical event identification channel: This channel is used to model potential risk events and symptoms contained in clinical texts, using clinical diagnosis and treatment text data As input, the original text sequence is first converted into is the word vector representation , then continuously through 5 chained Transformer encoding layers to perform contextual semantic modeling, capturing the potential event relationship and diagnostic semantic structure in the medical text; then through the fully connected layer to compress the data dimension and generate a unified dimensional data representation to obtain the output features of the clinical event recognition channel ;

[0083] 3. Behavioral compliance modeling channel: This channel is used to model the medication time series behavior characteristics and compliance change trends of contraceptive users, using medication time series data As input, we first use the GRU layer to capture the dynamic changes of short-term medication status and output behavior embedding ; Then the ESN layer is introduced to further model long-term behavioral dependencies and obtain long-term behavioral state representation , and generate the initial behavioral risk features through the Tanh activation function ;

[0084] To further enhance the perception of time-dependent structures, Repeatedly input the GRU-ESN-Tanh combination structure for deep dynamic modeling and output high-order behavioral features Finally, the feature dimensions are unified through the fully connected layer to generate the output features of the behavior compliance modeling channel ;

[0085] 4. Risk fusion prediction channel: Output features 、 、 Forming joint features through channel splicing ; Then the multi-head attention mechanism is introduced to achieve the saliency weighting between modalities and obtain the fusion representation , Input two multi-layer perceptron modules in sequence for nonlinear mapping and dimension compression, and generate the final fusion risk representation through the ReLU activation function ;

[0086] To achieve multi-label risk output, Input the following three sets of prediction heads respectively:

[0087] (1) Organ dysfunction prediction head: It consists of a fully connected layer and Sigmoid activation, and outputs the probability of organ dysfunction ;

[0088] (2) Adverse reaction type prediction head: consists of a fully connected layer and softmax activation, outputting the adverse reaction category ;

[0089] (3) Adverse reaction severity prediction head: It consists of a fully connected layer and Softmax activation, and outputs the severity classification result ;

[0090] above 、 and Together they constitute the multi-risk outputs of the model ;

[0091] 5. Model training and optimization: using a constructed multimodal risk dataset The model is trained and the Adam optimizer is used to update the parameters. When the preset maximum number of iterations is reached, the trained multimodal contraceptive risk prediction model is obtained.

[0092] 3. Risk Mechanism Modeling and Path Reasoning Module Design

[0093] In order to further enhance the model's understanding and modeling capabilities of the risk generation mechanism, the present invention collects multimodal input data. , and multiple risk output results Based on this, a risk mechanism modeling and path reasoning module is designed to comprehensively model the response relationship and propagation path between each modal input feature and the multi-risk prediction output, which specifically includes a modal response analysis submodule and a path reasoning submodule.

[0094] 1. Modal response analysis submodule: This submodule is used to quantify the contribution of each mode in the multi-risk prediction task and explore the synergistic interaction patterns between them. The specific method is as follows:

[0095] (1) Based on the trained multimodal contraceptive risk prediction model, input features of each modality in turn Perform independent prediction tests while keeping other pathways frozen; for example, when When testing modal data response, freeze the other two paths and obtain the Path prediction results of modal data ,and It includes the predicted probability of organ dysfunction, adverse reaction category prediction results, and severity classification results;

[0096] Therefore, based on this method, the independent response outputs of the three modes are obtained 、 、 ;

[0097] (2) To further quantify the real contribution of the modality to risk modeling, the independent output of the above modalities is combined with the complete model prediction output. , calculate the modal contribution :

[0098] ;

[0099] in, Indicates the contribution value of basic population information modality, Indicates the calculation of the L2 norm; this indicator reflects the degree of approximation of the overall prediction result by different structural information modes while maintaining the lack of other information. The higher the value, the stronger its independent prediction ability and the greater its contribution.

[0100] Afterwards, based on this contribution value calculation method, the contribution values ​​of clinical diagnosis and treatment text modality and medication time series modality are further obtained. 、 ; and form the modal contribution vector , which is used as the final output of the modal response analysis submodule.

[0101] 2. Path Reasoning Submodule: This submodule is used to construct a graph structure reflecting the risk propagation mechanism under unsupervised conditions and obtain the final risk propagation path based on task feedback reasoning, thereby reflecting the potential information flow leading to risk outcomes. The specific steps include:

[0102] (1) Based on structured demographic information , clinical diagnosis and treatment text information , and medication time series information , using the pre-trained ClinicalBERT medical language model to extract key medical information contained in each message and generate text keywords;

[0103] Keywords were obtained from structured demographic information: for example, “personal history of hypertension,” “family history of diabetes,” “having given birth,” and “age > 40.”

[0104] Keywords were obtained from clinical diagnosis and treatment text information, such as “irregular menstruation,” “drug-induced liver injury,” “left lower abdominal pain,” and “compound norethindrone tablet prescription.”

[0105] Obtain keywords from medication time series information: for example, “drug frequency ≥ 2 times / day,” “total dose exceeds the upper limit of treatment,” “missed dose ≥ 3 consecutive days,” etc.

[0106] Based on all the above Keywords get keyword set ,in Indicates the keyword text, and ;

[0107] (2) After that, treat each keyword as a node and Obtain the corresponding semantic features through the Word2Vec embedding layer , and use it as the feature of each keyword node;

[0108] (3) To construct the directed edge weights between keyword nodes in the mechanism graph, based on the S2 multi-risk output results , and the semantic features of keyword nodes, a risk response-guided directed edge weight calculation method was designed;

[0109] First, calculate the probability of organ dysfunction The average value of the organ risk intensity coefficient ; Prediction results for adverse reaction categories , take the probability value corresponding to the most likely adverse reaction category as the adverse reaction risk coefficient ; In addition, the results are classified according to the severity level The probability values ​​of severity levels 1 to 5 are weighted and summed (the corresponding weights for levels 1 to 5 are 0.2, 0.4, 0.6, 0.8, and 1.0, respectively) to obtain the severity level risk coefficient. ;

[0110] Afterwards, for any pair of keyword nodes in the graph, the embedding feature vectors of the two nodes are and Define edge direction weights as a basis for:

[0111] ;

[0112] in, and Respectively represent Hedi keyword node features, and ; To calculate the L2 norm, it is used to measure whether the change range of two keywords in the semantic space has potential cross-organ risks; To calculate the cosine similarity of the embedding vectors between two nodes, if the semantic relationship between the two keywords is highly consistent and the adverse reaction risk of the category is The higher it is, the stronger the relevance of the mechanism edge is; Indicates the activity level of the target keyword node and uses Weighting is performed to highlight the responsiveness of keywords with high severity;

[0113] Finally, the direction weights of all edges are calculated;

[0114] (4) The length of the preset risk transmission path is The larger the value, the longer the propagation path, which means that the path is allowed to contain more intermediate nodes and can cover richer mechanism association information; the smaller the value, the more accurate the high-edge-weight mechanism chain is, highlighting the core causal path;

[0115] Afterwards, take any keyword node in the mechanism diagram as the starting point and Under the constraint of , a greedy search algorithm is used to select the outgoing edge with the largest edge weight among the currently reachable nodes, gradually construct the path, and accumulate the path edge weights; finally, the path with the largest sum of path edge weights in the entire graph is extracted to form the optimal path for risk propagation. , for example: hormone intake - high frequency medication - metabolic abnormalities - elevated liver enzymes - abnormal liver function;

[0116] Finally, the output of the risk mechanism modeling and path reasoning module is the modal contribution vector and the optimal path for risk transmission .

[0117] 4. Comprehensive risk assessment and decision-making assistance methods

[0118] In order to fully integrate the output of multiple risk prediction results , and the resulting modal contribution vector and the optimal path for risk propagation , conduct comprehensive comprehensive assessment and auxiliary feedback, the present invention designs a comprehensive risk assessment and auxiliary decision prompting method; specifically includes the following steps:

[0119] 1. Output the three types of risk prediction results (probability of organ dysfunction , adverse reaction category , severity classification results ) as the basis for comprehensive risk assessment; at the same time, the modal contribution vector Extracted risk propagation paths As auxiliary explanatory variables, they constitute the multimodal evaluation input set;

[0120] 2. Based on Conduct comprehensive risk level assessment and jointly formulate four-level comprehensive risk assessment rules, including:

[0121] (1) Level I (low risk): probability of organ dysfunction , severity level , and the type of adverse reaction Common mild type (such as skin allergy, mild gastrointestinal reaction), indicating that there is basically no systemic impact and only routine monitoring is required;

[0122] (2) Level II (medium risk): probability of organ dysfunction , severity level ; or adverse reaction type Involvement of the digestive system or endocrine system indicates that vigilance and regular follow-up are recommended;

[0123] (3) Level III (high risk): probability of organ dysfunction ; or severity level ; or adverse reaction type Reactions involving the vascular system, urogenital system, and other central nervous systems indicate strong physiological effects and warrant proactive intervention and follow-up observation.

[0124] (4) Level IV (extremely high risk): severe level , and is accompanied by a high probability of organ dysfunction ( ) and high-risk adverse reaction types ( For drug-induced liver injury, cardiovascular risks, etc.), immediate intensive assessment and monitoring measures are required;

[0125] 3. Based on modal contribution vector and the optimal path for risk propagation Assist doctors in making decisions; specifically including:

[0126] (1) Select the modal contribution vector The system selects the modality with the highest contribution value and feeds back the result to the doctor to remind him to pay special attention. For example, when the imaging modality has the highest contribution value, the system generates the following prompt in the report: "Current risk prediction is mainly driven by clinical diagnosis and treatment texts. It is recommended to focus on relevant diagnosis and treatment information."

[0127] (2) Based on the optimal path of risk transmission Generate auxiliary explanations. For example, for the pathway of hormone intake-high-frequency medication-metabolism-elevated liver enzymes-abnormal liver function, generate the following reference data explanation: "The predicted result may be due to metabolic disorders caused by hormones, which have affected liver function. Please pay special attention to changes in liver indicators."

[0128] The final output is the comprehensive risk assessment results and the doctor's decision-making assistance prompts, thereby improving the clinical readability and decision-making support capabilities of the system output.

[0129] 5. Experimental Results and Analysis

[0130] 1. Multiple risk assessment experiments

[0131] To verify the predictive performance of the proposed multimodal risk prediction model under multiple risk indicators, three sets of task experiments were designed, corresponding to: organ dysfunction probability prediction, adverse reaction type classification, and adverse reaction severity classification. For each task, two classic models were set as comparison methods, and the most relevant modal information in the current task was used as input. The experiment used accuracy and stability as the core evaluation criteria, and conducted quantitative comparative analysis using standardized data sets and a unified evaluation process:

[0132] Task 1: Prediction of the probability of organ dysfunction

[0133] This task aims to predict the probability of dysfunction of five organ types (liver, kidney, cardiovascular, reproductive system, and nervous system), using mean square error (MSE) as the evaluation metric. The comparison methods include:

[0134] (1) LightGBM: takes structured population information as input and constructs an abnormal risk regression model based on a tree model;

[0135] (2) MedicalNet: takes urogenital system images as input, extracts image features and performs predictions;

[0136] The experimental results are as follows Figure 3 As shown in the figure, the experimental results show that the MSE performance of the proposed method is lower than that of the comparison method in all five organ types, especially in the nervous system and cardiovascular system, where the prediction accuracy is significantly improved. This verifies that the proposed method can better characterize the potential risks of individual organs through multi-modal joint modeling of structured basic information and diagnosis and treatment status, and improve the accuracy and robustness of abnormality recognition through multi-dimensional feature fusion.

[0137] Task 2: Classification of adverse reaction types

[0138] This task classifies adverse reactions into five major categories (based on the systems involved: gastrointestinal system reactions, endocrine disorders, skin and discoloration reactions, vascular system abnormalities, and nervous system problems). The evaluation indicator is Accuracy. The comparison method is:

[0139] (1) TextCNN: takes clinical diagnosis and treatment text as input and extracts risk clues based on local word windows for classification;

[0140] (2) GRU: Using medication time series as input, it models the role of changes in behavioral compliance in indicative of risk categories;

[0141] The experimental results are as follows Figure 4The results show that the method of the present invention has better accuracy than the comparison method in all adverse reaction categories, and the improvement is particularly obvious in the two highly heterogeneous categories of "vascular symptoms" and "gastrointestinal reactions", indicating that multimodal information modeling can more effectively capture potential abnormal manifestations and enhance the model's ability to discriminate adverse reaction risk types;

[0142] Task 3: Classification of severity of adverse reactions

[0143] This task is a multi-classification problem with 5 severity levels (WHO standard). Macro-F1 score is used for evaluation, focusing on the performance of the model in identifying high-risk levels (levels 4-5). The comparison methods include:

[0144] (1) BiLSTM: takes medication behavior sequence as input and captures time-dependent features;

[0145] (2) XGBoost: Using structured medical indicators as input, a regression tree classifier is constructed;

[0146] The experimental results are as follows Figure 5 The results show that the recognition effect of the present invention in the classification and prediction results of various risk levels is significantly better than that of the single-modal model, indicating that the joint modeling structure integrating behavioral characteristics, individual attributes and image information is more discriminative, can improve the perception of serious risk levels, and provide a more reliable basis for subsequent clinical early warning;

[0147] 2. Modal contribution analysis verification experiment

[0148] In order to evaluate the contribution of various modal information to the multi-risk prediction model proposed in this invention, a modal response ablation experiment was constructed. Without changing the network structure, the input modalities were "frozen" in turn to obtain the output deviation when different modalities independently participated in the prediction. The experiment selected 10 samples with complete data and calculated the L2 norm offset of each modality to the final prediction output as the contribution value indicator. The three types of modalities are: structured population information, clinical text information and medication time series; the experimental results are as follows Figure 6 As shown;

[0149] in Figure 6Each column represents a sample, and each row corresponds to the contribution of a modality. The experimental results show that different samples have obvious differences in the responses of each modal channel. Among them, clinical text has a higher contribution value in most samples, showing a stronger risk indication ability; while medication timing and structured information show a local dominant effect in some samples. This experiment fully verifies the effectiveness of the present invention in introducing modality-independent modeling channels and attention fusion strategies in model design, and provides a modality-level explanation basis for subsequent interpretability analysis; this experiment verifies the rationality of the present invention in introducing modality-independent channels and attention fusion mechanisms in the model structure, and provides a modality-level traceability basis for clinical explanatory output;

[0150] 3. Risk Path Reasonableness Scoring Experiment

[0151] To verify whether the "keyword-level risk transmission path" constructed by this invention has good clinical interpretability and reasoning rationality, three professional doctors with clinical experience were invited as scoring experts to subjectively score the risk transmission path automatically generated by the system based on the following three dimensions:

[0152] (1) Path logical coherence: whether a reasonable causal chain is formed between upstream and downstream nodes;

[0153] (2) Clarity of risk warnings: whether the pathway provides effective diagnostic or warning clues;

[0154] (3) Medical explanation support: whether the pathway is consistent with existing medical cognition and practical experience;

[0155] The scoring adopts a five-point system (1 point is extremely poor and 5 points is very good), and a total of 20 generation paths of real samples are evaluated.

[0156] Table 1 Risk path rationality scoring table

[0157]

[0158] The scoring results are shown in Table 1. The average score of the method of the present invention on the subjective five-point scoring standard was 4.41, among which the logical coherence score (4.52) was the highest, indicating that doctors generally recognized the structural rationality of the model generation pathway. The method also received high praise in terms of risk warning and medical interpretability, indicating that the mechanism pathway reasoning module proposed in the present invention can effectively output explanatory information with a logical reasoning chain and medical rationality, and has good potential for clinical decision-making assistance.

[0159] In summary, the method of the present invention demonstrates superior predictive ability in multiple risk tasks. Combining multimodal data fusion structure, mechanism modeling and interpretability module design, it not only improves the recognition accuracy and stability of the model, but also has good clinical practicality and readability, verifying its wide application value in the intelligent assessment of contraceptive safety.

[0160] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

[0161] Although the above describes the specific implementation methods of the present invention, it does not limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A multimodal risk assessment and decision-making support method for contraceptive safety monitoring, characterized in that: The following processes are included: S1, collects multimodal medical data related to contraceptive use, including structured demographic information, clinical diagnosis and treatment texts, and medication time series data; S2, inputs the collected data in S1 into the trained contraceptive multi-risk prediction model. The model includes three independent modal channels, which respectively encode and model basic demographic information, clinical text, and medication behavior sequences, and fuse them to generate a unified representation for predicting and outputting multiple key risk indicators, including the probability of organ dysfunction, adverse reaction type, and adverse reaction severity level; S3, based on the contraceptive pill risk prediction model trained in S2, performs independent prediction tests on each modal input feature in turn, and calculates the joint prediction results of the three modalities to obtain the modal contribution vector; Based on structured demographic information, clinical diagnosis and treatment text information, and medication time series information, key medical information contained in each information is extracted to construct the optimal risk transmission path; S4 generates a comprehensive risk assessment result based on the prediction results of S2 and the set risk level judgment rules; at the same time, it provides doctors with auxiliary decision-making prompts based on the modal contribution vector and the optimal risk propagation path information.

2. The multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 1, characterized in that: The multimodal medical data specifically includes: Structured demographic information: including basic medical background variables such as age, gender, reproductive history, past medical history, and family medical history of contraceptive users; and organizing and splicing this information into a basic demographic information vector ; Clinical diagnosis and treatment text information: including unstructured text such as doctor's consultation notes, diagnosis results, prescription information, and adverse reaction descriptions ; Medication time series information: records the medication use records of contraceptive users in each time period during the entire use cycle, including: drug name and type, manufacturer number, production batch number, medication dosage, medication frequency, and missed or missed doses; thus obtaining the medication time series data of contraceptive users ,and ;in, Indicates in Medication records for a period of time, and .

3. The multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 1, characterized in that: The probability of organ dysfunction includes expert scoring of the probability of liver and kidney damage, cardiovascular dysfunction, reproductive system abnormality, and nervous system dysfunction to obtain an organ dysfunction vector label; The adverse reaction types are coded according to common clinical types, including gastrointestinal system reactions, endocrine disorders, skin and discoloration reactions, vascular system abnormalities, nervous system problems, and other reactions, to obtain adverse reaction coding vector labels; The severity level of the adverse reaction is labeled from 1 to 5 according to the WHO international standard, and the higher the score, the more severe the adverse reaction.

4. The multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 1, characterized in that: The contraceptive pill multi-risk prediction model includes a basic risk modeling channel, a clinical event identification channel, and a behavioral compliance modeling channel; The basic risk modeling channel is based on the basic population information vector As input, we get the risk background characteristics ; The clinical event identification channel is based on clinical diagnosis and treatment text data As input, we get the output features of the clinical event recognition channel ; The behavioral compliance modeling channel is based on drug time series data As input, the output features of the generated behavior compliance modeling channel are obtained ; Finally, the output features will be 、 、 Forming joint features through channel splicing ; Then the multi-head attention mechanism is introduced to achieve the saliency weighting between modalities and obtain the fusion representation , Input two multi-layer perceptron modules in sequence for nonlinear mapping and dimension compression, and generate the final fusion risk representation through the ReLU activation function ; Final fusion risk representation Input three sets of prediction heads and output the probability of organ dysfunction, adverse reaction type and adverse reaction severity level.

5. A multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 4, characterized in that: The basic risk modeling channel first performs nonlinear feature expansion through two serially connected KAN function encoding layers, where each KAN function encoding layer consists of a base function layer, a projection mapping layer based on a fully connected layer, and a dropout layer, which is used to accurately fit and combine different types of structural variables. Then the fitting is strengthened again through the KAN function encoding layer, and then the nonlinear expression is enhanced through the SiLU activation function to obtain the primary structural risk vector , and use the ResNet residual structure to convert the original input and Splicing is performed to obtain the output features of the first set of double-layer KAN-SiLU-Resnet combination ; Then, The high-order feature combination is performed again through the double-layer KAN-SiLU-Resnet combination, and the output feature is obtained ; Finally, through the fully connected layer Map to a unified fusion dimension and obtain risk background characteristics .

6. A multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 4, characterized in that: The clinical event recognition pipeline first converts the original text sequence into is the word vector representation , then continuously through 5 chained Transformer encoding layers to perform contextual semantic modeling, capturing the potential event relationship and diagnostic semantic structure in the medical text; then through the fully connected layer to compress the data dimension and generate a unified dimensional data representation to obtain the output features of the clinical event recognition channel .

7. The multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 4, characterized in that: The behavioral compliance modeling pipeline first uses the GRU layer to capture the dynamic changes of short-term medication status and outputs the behavioral embedding ; Then the ESN layer is introduced to further model long-term behavioral dependencies and obtain long-term behavioral state representation , and generate the initial behavioral risk features through the Tanh activation function ; Repeatedly input the GRU-ESN-Tanh combination structure for deep dynamic modeling and output high-order behavioral features Finally, the feature dimensions are unified through the fully connected layer to generate the output features of the behavior compliance modeling channel .

8. The multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 1, characterized in that: The specific process of obtaining the modal contribution vector in S3 is: Based on the trained contraceptive risk prediction model, each modal input feature is tested independently in turn. During the test, other pathways are frozen to obtain independent response outputs of the three modalities. 、 、 ; Based on combining the above modal independent outputs with the complete model to predict the output , calculate the modal contribution ; First, the mode-independent outputs are calculated , which is 、 、 One of the values ​​is combined with the complete model to predict the output The L2 norm of the difference between The L2 norm of the calculation result is normalized to obtain the calculation result, and the inverse of the calculation result is taken as the contribution value of the mode. .

9. The multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 1, characterized in that: The specific process of constructing the optimal risk propagation path in S3 is as follows: S31, based on structured demographic information , clinical diagnosis and treatment text information and medication time series information , using the pre-trained ClinicalBERT medical language model to extract key medical information contained in each message and generate text keywords; Based on all Keywords get keyword set ,in Indicates the keyword text, and ; S32, each keyword is regarded as a node, and the keyword text Obtain the corresponding semantic features through the Word2Vec embedding layer , and use it as the feature of each keyword node; S33, calculate the probability of organ dysfunction The average value of the organ risk intensity coefficient ; Prediction results for adverse reaction categories , take the probability value corresponding to the most likely adverse reaction category as the adverse reaction risk coefficient ; In addition, the results were classified according to severity level. The probability values ​​of severity levels 1 to 5 are weighted and summed to obtain the severity level risk coefficient. ; For any pair of keyword nodes in the graph, the embedding feature vectors of the two nodes are and Define edge direction weights as a basis ; Calculate the direction weights of all edges; and Respectively represent Hedi keyword node features, and ; S34, the length of the preset risk transmission path is , starting from any keyword node in the mechanism diagram, Under the constraint of , a greedy search algorithm is used to select the outgoing edge with the largest edge weight among the currently reachable nodes, gradually construct the path, and accumulate the path edge weights; finally, the path with the largest sum of path edge weights in the entire graph is extracted to form the optimal path for risk propagation. .

10. The multimodal risk assessment and decision-making support method for contraceptive safety monitoring according to claim 1, characterized in that: The probability of organ dysfunction in S4 , adverse reaction category , severity classification results A four-level comprehensive risk assessment rule has been formulated, including: Level I low risk: probability of organ dysfunction , severity level , and the type of adverse reaction Common mild type, meaning there is basically no systemic impact and only routine monitoring is required; Level II medium risk: probability of organ dysfunction , severity level ; or adverse reaction type Involvement of the digestive system or endocrine system indicates that vigilance and regular follow-up are recommended; Level III high risk: probability of organ dysfunction ; or severity level ; or adverse reaction type For central nervous system reactions, active intervention and follow-up observation are required; Level IV Very high risk: severe level , and is accompanied by a high probability of organ dysfunction , and high-risk adverse reaction types require immediate intensive assessment and monitoring measures.

Images