A breast reconstruction procedure recommendation system and method based on multi-modal data
By integrating multimodal data and performing nonlinear analysis, the problem of inaccurate risk assessment in breast reconstruction procedure recommendations has been solved, enabling scientific quantification and individualized matching of procedure selection and enhancing the ability to cope with uncertainty.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TIANJIN TUMOR HOSPITAL
- Filing Date
- 2026-04-03
- Publication Date
- 2026-07-14
AI Technical Summary
Current recommendations for breast reconstruction techniques rely on linear summation assessments of single indicators, which cannot quantify the complex nonlinear mechanisms of metabolic and vascular damage, resulting in inaccurate risk assessments and a lack of targeted recommendations.
A breast reconstruction procedure recommendation system based on multimodal data is adopted. Through the feature calculation and analysis module, clinical information, metabolic vascular injury index calculation, imaging features and test features are integrated. By using nonlinear fusion and interaction effect analysis, a continuous numerical risk index is generated. Combined with the multimodal feature weighted fusion algorithm, the system can accurately predict and dynamically adjust the surgical risk.
It accurately captures the nonlinear cumulative effects of metabolism and vascular damage, distinguishes the true risk differences among different patients, and achieves scientific quantification and individualized matching of surgical procedures. This enhances the ability to cope with uncertainty and avoids missed judgments and overtreatment in traditional assessments.
Smart Images
Figure CN122393017A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of breast reconstruction procedure recommendation technology, and more specifically, to a breast reconstruction procedure recommendation system and method based on multimodal data. Background Technology
[0002] Breast reconstruction is a key medical technology performed after mastectomy for patients with breast cancer and other diseases to restore breast shape and function and improve their quality of life. Surgical procedure recommendation is a core aspect of this field. It requires a comprehensive consideration of factors such as the patient's physical condition, pathological characteristics, and anatomical structure to match the patient with a safe, effective, and highly adaptable individualized surgical plan, which directly affects the success rate of the surgery, the incidence of postoperative complications, and the patient's long-term recovery.
[0003] In current clinical practice, recommendations for breast reconstruction procedures mainly rely on two approaches: one is based on the physician's clinical experience, combined with the patient's basic medical history, simple laboratory indicators, and imaging data to make a subjective judgment; the other is to use preliminary digital tools to provide surgical recommendations after simply integrating limited patient data.
[0004] Specifically, existing technologies often rely on single or a few indicators, such as the duration of chronic disease, single blood glucose test values, and basic breast morphology, to assess surgical risk and select recommended surgical procedures through linear aggregation. This approach fails to adequately consider the complex pathological mechanisms of metabolic and vascular damage. The cumulative effect of metabolic abnormalities and vascular damage is not a simple additive relationship but exhibits a significant non-linear correlation, with synergistic damaging effects among different risk factors. For example, using only disease duration as an assessment criterion makes it difficult to distinguish the true risk difference between patients with "short disease duration but extremely poor blood glucose control" and "long disease duration but good blood glucose control," easily leading to underestimation of high-risk patients or overtreatment of low-risk patients, resulting in unscientific and untargeted surgical recommendations. Therefore, we propose a breast reconstruction surgical procedure recommendation system and method based on multimodal data. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the prior art, adapt to practical needs, and provide a breast reconstruction procedure recommendation system and method based on multimodal data. This addresses the technical problem that traditional procedure recommendations rely on linear superposition assessment of a single indicator, which cannot quantify the nonlinear and complex mechanisms of metabolic and vascular damage, resulting in inaccurate risk assessment and a lack of targeted recommendations.
[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a breast reconstruction procedure recommendation system based on multimodal data, the system including a feature calculation and analysis module, the feature calculation and analysis module including a clinical information structuring unit, a metabolic vascular injury index calculation module, an image feature unit, and a test feature unit; The clinical information structured unit is configured to extract and quantify clinical features, including the course of chronic diseases and glycemic control levels, from patient clinical text data. The metabolic vascular injury index calculation module is connected to the information structuring unit and includes an input processing unit, a core calculation unit, and an output generation unit. The input processing unit is configured to receive quantified chronic disease course and blood glucose control level from the clinical information structuring unit, and to receive at least one comorbidity risk factor from the clinical information structuring unit. The core computing unit is configured to perform nonlinear fusion and interaction effect analysis on the received chronic disease course, blood glucose control level and comorbidity risk factors through a pre-trained computing model, and generate a continuous numerical risk index characterizing the degree of metabolic and vascular cumulative damage in patients. The output interface unit is configured to generate and output a continuous numerical risk index characterizing the degree of cumulative damage to the patient's metabolic and vascular systems based on the analysis results of the core computing unit. The image feature unit is configured to extract anatomical and physiological features related to breast reconstruction surgery from medical image data. The test feature analysis unit is configured to process and analyze the patient's laboratory test data.
[0007] Preferably, the nonlinear fusion and interaction effect analysis of the core computing unit is achieved through the following formula: ; in, The metabolic vascular damage index has a value range of 0-1. The Sigmoid nonlinear transformation function has the following core expression: ; The course of the disease is chronic; This is a nonlinear normalization function for the disease course; Blood sugar control level; A standardized function for blood glucose control levels; For the first Comorbidity risk factors; A function to identify the comorbidity status; , ,and These are the weighting coefficients for disease duration, glycemic control level, and comorbidities, respectively, satisfying... ; For the first Comorbidities and the first Interaction weights between comorbidities; This is the model calibration bias term, with a value range of [value range missing]. .
[0008] Preferably, it also includes a multimodal data acquisition module, which comprises a text data unit, a test data unit, an image data unit, and a 3D morphology analysis unit; The text data unit is used for electronic medical record mining, medical history information extraction, and clinical report parsing; The test data unit is used to process test indicator data such as blood glucose series and HbA1c sequence. The image data unit is used to analyze breast-related image data such as CT / CTA, MRI, and ultrasound. The 3D morphological analysis unit is used to collect and preprocess the patient's 3D body surface morphological data, which includes breast volume, three-dimensional contour symmetry, and body surface topological features.
[0009] Preferably, it also includes an intelligent decision-making and prediction module, which includes a multimodal fusion decision-making unit and an intraoperative dynamic adjustment unit; The multimodal fusion decision unit is connected to the feature calculation and analysis module and is configured to receive a continuous numerical risk index from the metabolic vascular injury index calculation module, anatomical structure features and physiological state features from the image feature unit, and test index features from the test feature unit. Using the continuous numerical risk index as a priori risk guide, the unit performs weighted fusion on the received multimodal features and predicts the probability of various complications for different breast reconstruction procedures based on the fused features. The intraoperative dynamic adjustment unit is configured to receive real-time intraoperative exploration data, trigger the updating of features output by the image feature unit or the verification feature unit based on the exploration data, and notify the multimodal fusion decision unit to re-predict based on the updated features.
[0010] Preferably, the weighted fusion of the multimodal fusion decision unit is achieved through the following formula: Multimodal feature weighted fusion formula: ; Formula for predicting the probability of complications: ; in, This is a multimodal fusion feature vector; for The guiding weight of the index; These are the weighting coefficients for image features; This represents the normalized image feature vector; To normalize and test the eigenvectors; For specific surgical procedure numbers; For the first The probability of complications associated with this surgical procedure; For the first Feature weight matrix of the technique; This represents the total number of available breast reconstruction procedures.
[0011] Preferably, it also includes a result output and application module, which includes an intraoperative recommendation visualization unit, an interpretable report generation unit, and a clinical management key points unit; The intraoperative recommendation visualization unit is configured to receive and display updated surgical risk assessments and adjustment suggestions in real time during the operation. The interpretable report generation unit is configured to generate an interpretable report containing a decision logic chain, which is used to clarify the correlation between the continuously numerical risk index output by the metabolic vascular injury index calculation module and the fact that a specific surgical procedure is marked as high-risk or recommended. The clinical management key points unit is configured to automatically match and generate a personalized list of perioperative management recommendations based on a continuous numerical risk index and the selected surgical procedure.
[0012] Preferably, it also includes a system optimization and learning module, which includes a clinical outcome collection unit and a model optimization iteration unit; The clinical outcome collection unit is configured to collect and structure-store real-world clinical outcome data related to the patient's breast reconstruction surgery, including at least the actual surgical record, postoperative complications, and long-term follow-up results. The model optimization iteration unit is connected to the clinical results collection unit and the feature calculation and analysis module. It is configured to periodically or according to triggering conditions use new data collected by the clinical results collection unit to incrementally learn or retrain the pre-trained calculation model in the metabolic vascular injury index calculation module to optimize the calculation accuracy of the continuous numerical risk index.
[0013] Preferably, the incremental learning of the model optimization iteration unit updates the model weights using the following formula: ; The loss function is defined as: ; in, This is the updated model weight vector; For the course of chronic diseases The weighting coefficients, Blood sugar control level The weighting coefficients, For the first Comorbidity risk factors The weighting coefficients, For the first Comorbidities and the first Interaction effect weights among comorbidities; The model weight vector before the update; The learning rate; For loss function For weight vector The gradient; The loss function; For the first one sample Predicted value; For the first one sample The actual value; is the L2 regularization coefficient.
[0014] A method for recommending breast reconstruction procedures based on multimodal data, comprising the following steps: S100: Simultaneously acquires patient text, laboratory, imaging, and 3D morphological data through the multimodal data acquisition module, completes preprocessing and structured conversion, and outputs the patient's medical history. Blood sugar control level Comorbidities Original image features and original examination features; The S200 module, along with the feature calculation and analysis module, extracts features from the acquired data. The core module generates the MVI index through the metabolic vascular injury index calculation module, while simultaneously outputting normalized image features. and normalization test characteristics Accurately distinguish patient risks under different control states; The S300 and intelligent decision-making and prediction modules generate multimodal fusion features through a weighted fusion formula. Then, after passing through the probability normalization function Calculate each technique Output the initial recommended solution; S400, dynamically updating normalized image features based on intraoperative exploration data. and normalization test characteristics Recalculate multimodal fusion features With probability normalization function Calculate each technique Optimize recommendation schemes in real time; S500 provides visual recommendations, interpretable reports, and management suggestions through results output and application modules; S600: Collect postoperative clinical results and update model weights through incremental learning formulas to achieve continuous optimization of system performance. Compared with the prior art, the beneficial effects of the present invention are: 1. This invention integrates nonlinear normalization functions, standardized functions, interaction effect analysis algorithms, and the sigmoid function through a metabolic vascular injury index calculation module. This allows for in-depth analysis of the complex relationships between disease duration, glycemic control levels, and comorbidities. This design accurately captures the nonlinear cumulative effect of metabolic and vascular injury, effectively distinguishing the true risk difference between patients with short disease duration but extremely poor glycemic control and those with long disease duration but good glycemic control. This makes risk assessment more aligned with pathophysiological patterns, providing a scientific quantitative basis for surgical procedure selection. It addresses the problem that traditional surgical recommendations rely on linear superposition assessments of single indicators, failing to quantify the complex nonlinear mechanisms of metabolic and vascular injury, leading to inaccurate risk assessments and a lack of targeted recommendations.
[0015] 2. This invention also utilizes a multimodal feature weighted fusion algorithm and the Softmax function to accurately predict the probability distribution of complications associated with different breast reconstruction procedures based on multi-dimensional data such as the MVI index, imaging features, and diagnostic features. For patients with multiple underlying diseases such as diabetes and hypertension, the system can specifically calculate the suitability risk of each procedure, clearly define the boundaries between high-risk and low-risk procedures, and avoid increased surgical risks due to synergistic damage from comorbidities, thus achieving precise matching of procedures based on individual risk characteristics.
[0016] 3. This invention also receives real-time intraoperative exploration data, such as new findings on tissue blood supply, through an intraoperative dynamic adjustment unit, and immediately triggers the system to update relevant features, notifying the multimodal fusion decision unit to re-perform weighted fusion and risk probability calculation. This means that the generated accurate MVI index and the established probability prediction model are not static but can be dynamically updated as the surgery progresses. This directly elevates preoperative static assessment to intraoperative dynamic navigation. When the patient's local vascular conditions are found to be better or worse than expected during the operation, the system can immediately optimize recommendations, making the surgical procedure selection more in line with the actual intraoperative situation and greatly enhancing the ability to cope with uncertainties. Attached Figure Description
[0017] Figure 1 This is a system block diagram of the present invention; Figure 2 This is a specific system block diagram of the intelligent decision-making and prediction module of the present invention; Figure 3 This is a flowchart of the method of the present invention. Detailed Implementation
[0018] Example 1: like Figures 1 to 2As shown, this embodiment provides a breast reconstruction procedure recommendation system based on multimodal data. The system includes a multimodal data acquisition module, which is used to comprehensively collect multi-dimensional data related to the patient and breast reconstruction surgery, providing a complete data source for subsequent feature calculation, including text data unit, test data unit, image data unit, and 3D morphological analysis unit. The text data unit is used for electronic medical record mining, medical history information extraction, and clinical report parsing. It uses natural language processing technology to perform structured mining on electronic medical records, medical history records, and clinical reports, extracting text information such as patients' chronic disease history (e.g., diabetes, hypertension), surgical history, and allergy history, and converting it into quantifiable structured data. The test data unit is used to process test indicator data such as blood glucose series and glycated hemoglobin (HbA1c) sequence, and connects to the laboratory information system (LIS) to collect and process test data such as blood glucose series indicators, glycated hemoglobin (HbA1c) sequence, liver and kidney function, and coagulation function. It interpolates and completes missing values and standardizes outliers. Among them, glycated hemoglobin (HbA1c) is the core indicator reflecting the average blood glucose level over the past 2-3 months and is a key basis for assessing the quality of blood glucose control. The image data unit is used to analyze breast-related image data such as CT scans (CT) / CTA, magnetic resonance imaging (MRI), and ultrasound. It first connects to the medical imaging archive and communication system (PACS) to obtain the patient's breast CT / CTA, MRI, ultrasound, and other image data. The data is then preliminarily processed through image segmentation and feature extraction algorithms to lay the foundation for subsequent anatomical and physiological feature extraction. For patients scheduled for free flap reconstruction, the CTA data can be used to accurately analyze the vascular distribution characteristics and reduce the risk of intraoperative vascular anastomosis. The 3D morphological analysis unit collects 3D morphological data of the patient's breast and chest wall region through a 3D body surface scanning device. After preprocessing, quantitative indicators such as breast volume, three-dimensional contour symmetry, and body surface topological features are extracted to provide morphological basis for surgical matching and ensure the consistency of the reconstructed breast with the healthy side.
[0019] Furthermore, it also includes a feature calculation and analysis module, which includes a clinical information structuring unit, a metabolic vascular injury index calculation module, an imaging feature unit, and a test feature unit; The clinical information structuring unit is configured to extract and quantify clinical features, including the duration of chronic disease and blood glucose control level, from the patient's clinical text data, receive the output results of the text data unit, quantify and encode clinical information such as the duration of chronic disease (e.g., the duration of diabetes), blood glucose control level, and mean glycated hemoglobin (HbA1c), and output standardized clinical feature vectors, such as the quantified value of duration T and the quantified value of blood glucose control G. The metabolic vascular injury index calculation module is connected to the information structuring unit and includes an input processing unit, a core calculation unit, and an output generation unit. The input processing unit is configured to receive quantified chronic disease duration and glycemic control level from the clinical information structuring unit, and to receive at least one comorbidity risk factor from the clinical information structuring unit. The received data includes the quantified chronic disease duration (T), directly quantified from the duration extracted from clinical text; the glycemic control level (G), with mean HbA1c as the core quantification indicator; and at least one comorbidity risk factor. For example, hypertension and hyperlipidemia are represented by binary variables: 1 = present and 0 = absent. The core computing unit is configured to perform nonlinear fusion and interaction effect analysis on the received chronic disease course, blood glucose control level and comorbidity risk factors through a pre-trained computing model, and generate a continuous numerical risk index characterizing the degree of metabolic and vascular cumulative damage in patients. The pre-trained computational model uses nonlinear fusion and interaction effect analysis to generate a continuous numerical risk index. The core algorithm formula is as follows: ; in, The metabolic vascular injury index is used to quantify the degree of cumulative damage to a patient's metabolic and vascular systems. The value ranges from 0 to 1, where 0 represents no significant damage and 1 represents extremely severe damage. The Sigmoid nonlinear transformation function has the following core expression: Its function is to map the linear result after multi-parameter fusion to a continuous interval of 0-1 to avoid numerical overflow, while adapting to the application scenario of clinical risk grading and quantification. For the course of chronic diseases The weighting coefficient, determined through clinical data fitting, is 0.25 (satisfying...). This reflects the fundamental contribution of the disease course to vascular damage; The course of the disease is chronic; This is a nonlinear normalization function for the disease course, with the core expression being: ; Blood sugar control level; Blood sugar control level The weighting coefficients, and Together, they constitute the core indicator weights, ensuring the decision-making dominance in blood glucose control quality; A standardized function for blood glucose control levels; For the first Comorbidity risk factors; For the first Comorbidity risk factors The weighting coefficients are dynamically adapted to the type of comorbidity. This is a function for identifying the comorbidity status; its core expression is: The binary variable (1 = present, 0 = absent) is directly mapped to the damage contribution value. If it exists, the corresponding weight is added; if it does not exist, there is no contribution. For the first Comorbidities and the first The interaction effect weights among comorbidities are optimized by training with clinical data. Index for the first comorbidity (corresponding) =High blood pressure, =High blood lipids, =Kidney disease, etc.), For the index of the second comorbidity, it must meet the following requirements. such as the course of the disease With hypertension Interaction weights Disease course and hyperlipidemia Interaction weights It is used to quantify the synergistic amplification effect of basic damage and comorbidities, such as how hypertension can exacerbate vascular endothelial inflammation caused by hyperglycemia, thus doubling the rate of damage. This is the model calibration bias term, with a value range of [value range missing]. Determined by multi-center data validation, it is used to correct detection errors from different clinical centers, such as model bias caused by differences in HbA1c detection methods; in, The design logic is as follows: In the early stages of the disease (0-5 years), the function increases rapidly, aligning with the pathological pattern of rapid progression of vascular damage in the early stages of metabolic abnormalities; after 10 years, the function value tends to level off, such as... hour, , hour, To avoid misjudgment of risks caused by overweighting of long disease courses; exist In the above, 5.7% is the normal borderline value for HbA1c, and 3.3 is the span coefficient between the normal range (4.0%-5.7%) and the pathological high-value range (5.7%-9.0%). The function logic is: when... hour (No damage contribution) hour (Contribution of severe injury). hour (Contribution of extremely severe damage), accurately capturing the threshold damage effect of hyperglycemia; a. Nonlinear adaptive physiological laws: through Nonlinear normalization of the disease course is applied to avoid overweighting for long disease durations. For example, when the disease duration exceeds 10 years, the function growth tends to level off, consistent with the clinical pattern of slower damage rate after a certain disease duration. Standardizing HbA1c to the 0-1 range allows for precise adaptation to the threshold effect of blood glucose levels. For example, if HbA1c exceeds 9%, vascular damage accelerates. The value rapidly approaches 1, significantly improving sensitivity to hyperglycemic damage; b. Quantifying synergistic damage effects: Interaction term: It can accurately capture the synergistic damage of disease course + comorbidities, blood glucose + comorbidities, such as the superimposed effect of diabetes course and hypertension. Hypertension will aggravate vascular endothelial damage caused by hyperglycemia. This interaction item can quantify this clinical association into a numerical contribution, which is more realistic than single indicator assessment. c. Achieving Precise Risk Differentiation: Through weight allocation and function mapping, the risk differences between patients with short disease duration but poor control and those with long disease duration but good control can be effectively differentiated. For example, a patient with a 3-year disease duration but extremely poor glycemic control (HbA1c > 9%)... , The contribution value is higher after weighting; while patients with a disease duration of 5 years but good HbA1c control of <7% have a higher contribution value. , Ultimately, the former The index will be higher than the latter, which is more accurate than simply looking at the course of the disease, and solves the logical flaw of traditional assessment that emphasizes the course of the disease and neglects control. Output normalization: The Sigmoid function ensures The index remains stable in the 0-1 range. Higher values indicate more severe cumulative metabolic and vascular damage, facilitating rapid clinical classification of low-risk individuals. Medium risk High risk Three levels; The output interface unit outputs the final metabolic vascular damage index based on the analysis results of the core computing unit. As the core prior risk basis for subsequent decision-making, it provides a quantitative benchmark for surgical procedure selection; By integrating nonlinear normalization functions, standardized functions, interaction effect analysis algorithms, and the sigmoid function through the metabolic vascular injury index calculation module, the complex relationships between disease duration, glycemic control levels, and comorbidities can be deeply analyzed. This design can accurately capture the nonlinear cumulative effect of metabolism and vascular injury, effectively distinguishing the true risk difference between patients with short disease duration but extremely poor glycemic control and those with long disease duration but good glycemic control. This makes risk assessment more aligned with pathophysiological patterns, providing a scientific quantitative basis for surgical procedure selection. It solves the problem that traditional surgical procedure recommendations rely on linear superposition assessments of single indicators, which cannot quantify the complex nonlinear mechanisms of metabolism and vascular injury, leading to inaccurate risk assessments and a lack of targeted recommendations.
[0020] The image feature unit is configured to extract anatomical and physiological features related to breast reconstruction surgery from medical image data, receive the preprocessing results of the image data unit, and extract breast anatomical features, such as blood vessel distribution density, gland thickness, chest wall bone morphology and physiological features, such as tissue perfusion status, through deep learning algorithms to form a standardized image feature vector. The test feature analysis unit is configured to process and analyze the patient's laboratory test indicator data, receive the processing results from the test data unit, perform quantitative analysis and multi-dimensional feature fusion on indicators such as blood glucose series, HbA1c sequence, and liver and kidney function, and output test feature vectors to provide multi-dimensional support for risk assessment.
[0021] Furthermore, it also includes an intelligent decision-making and prediction module, which realizes surgical risk prediction and dynamic adjustment based on feature calculation results. The intelligent decision-making and prediction module includes a multimodal fusion decision unit and an intraoperative dynamic adjustment unit. The multimodal fusion decision unit is connected to the feature calculation and analysis module and is configured to receive a continuous numerical risk index from the metabolic vascular injury index calculation module, anatomical structure features and physiological state features from the image feature unit, and test index features from the test feature unit. Using the continuous numerical risk index as a priori risk guide, the unit performs weighted fusion on the received multimodal features and predicts the probability of various complications for different breast reconstruction procedures based on the fused features. The multimodal fusion decision unit and the feature calculation and analysis module are fully connected, receiving... The index, image feature vector, and test feature vector are used to achieve technique matching through weighted fusion and probabilistic prediction. The core algorithm is as follows: Multimodal feature weighted fusion formula: ; in, This is a multimodal fusion feature vector that integrates risk indicators with physiological and diagnostic features; for The index's guiding weight, ranging from 0.4 to 0.6, establishes its core decision-making role in metabolic vascular injury. The weighting coefficient for image features ranges from 0.5 to 0.7, highlighting the influence of anatomical structure and physiological state on the suitability of surgical procedures. The normalized image feature vector is obtained by normalizing the original features output by the image feature unit through Min-Max normalization and mapping it to the 0-1 interval. The normalized test feature vector is obtained by standardizing the original features output by the test feature unit using Z-score, with a mean of 0 and a variance of 1. pass Weights ensure Lead the risk assessment logic and avoid non-critical features interfering with core decisions; pass The weighting emphasizes imaging features, such as vascular distribution and breast morphology, and their surgical suitability, balancing the dual needs of risk control and morphological matching. This addresses the issue of low prediction accuracy for single-modal features, for example, for... In other words, for patients at medium risk, if imaging features show a dense distribution of blood vessels, High values in relevant dimensions can increase the recommendation priority for free flap reconstruction. Formula for predicting the probability of complications: ; in, For the first The probability of complications for this surgical procedure is measured in values ranging from 0 to 1, with lower values indicating higher safety. This is a probability normalization function used to convert the linear output of the fused features into the risk probability distribution of each technique; For the first The feature weight matrix of each surgical procedure is trained from clinical samples and is adapted to the risk characteristics of different surgical procedures. For the first The bias term of this technique is used to correct the systematic bias of the weight matrix; The total number of available breast reconstruction procedures, such as implant surgery, latissimus dorsi myocutaneous flap reconstruction, and abdominal free flap reconstruction, is dynamically adjusted based on the clinically commonly used surgical procedure database. For specific procedure numbers, For prosthesis implantation, For latissimus dorsi myocutaneous flap reconstruction, 3 is abdominal free flap reconstruction, and This is a pedicled flap transplantation procedure; pass The function maps the fused features to the complication risk probability distribution of each surgical procedure, ensuring that the sum of the risk probabilities of all procedures is 1, allowing clinicians to directly select the appropriate procedure. The minimum recommended approach is 1-2 surgical techniques; for example, for For high-risk patients, the formula will calculate the cost of free flap reconstruction. The risk of vascular crisis is significantly higher than that of implant surgery, thus automatically avoiding high-risk surgical procedures; This unit is based on The procedure is sorted and selected to identify 1-2 recommended surgical methods with the lowest risk of complications. The core risk points of each procedure are also output, such as prosthesis implantation: infection risk P=0.05. By employing a multimodal feature weighted fusion algorithm and the Softmax function, this invention can accurately predict the probability distribution of complications associated with different breast reconstruction procedures based on multi-dimensional data such as the MVI index, imaging features, and diagnostic features. For patients with multiple underlying diseases such as diabetes and hypertension, the system can specifically calculate the suitability risk for each procedure, clearly define the boundaries between high-risk and low-risk procedures, and avoid increased surgical risks due to synergistic damage caused by comorbidities, thus achieving precise matching of procedures based on individual risk characteristics.
[0022] The intraoperative dynamic adjustment unit is configured to receive real-time intraoperative exploration data, trigger the updating of features output by the image feature unit or the verification feature unit based on the exploration data, and notify the multimodal fusion decision unit to re-predict based on the updated features.
[0023] Furthermore, it also includes a result output and application module, which transforms the decision results into clinically usable visual information and management suggestions. The result output and application module includes an intraoperative recommendation visualization unit, an interpretable report generation unit, and a clinical management key points unit. The intraoperative recommendation visualization unit is configured to receive and display updated surgical risk assessments and adjustment suggestions in real time during the operation. Through a display terminal in the operating room, it displays the updated list of recommended surgical procedures and the probability of complications for each procedure in real time. , The index and key influencing factors, such as HbA1c=9.2% corresponding to The risk level is increased by 0.2, and the information is presented in the form of bar charts, risk heat maps, and text, making it easy for surgeons to quickly grasp the core information. The intraoperative dynamic adjustment unit receives real-time intraoperative exploration data, such as new findings on tissue blood supply, and immediately triggers the system to update relevant features, notifying the multimodal fusion decision unit to recalculate weighted fusion and risk probability. This means that the generated accurate MVI index and the established probability prediction model are not static but can be dynamically updated as the surgery progresses. This directly elevates preoperative static assessment to intraoperative dynamic navigation. When the patient's local vascular conditions are found to be better or worse than expected during the operation, the system can instantly optimize recommendations, making the surgical procedure selection more aligned with the actual intraoperative situation and greatly enhancing the ability to cope with uncertainties.
[0024] The interpretable report generation unit is configured to generate an interpretable report containing a decision logic chain, which is used to clarify the association between the continuous numerical risk index output by the metabolic vascular injury index calculation module and specific surgical procedures that are marked as high risk or recommended. Automatically generate an explanatory report containing the decision logic chain, clearly clarifying... The correlation between the index and the risk level of each surgical technique, such as: (High-damage) corresponding to the risk of complications in pedicled flap transplantation (High-risk) Corresponding recommended procedure: implantation of a prosthesis. This enhances decision-making transparency and helps surgeons clearly explain the basis for recommendations to patients. The clinical management key points unit is configured to be based on The index is matched with the selected surgical procedure and a pre-defined clinical pathway database to generate a personalized list of perioperative management recommendations, including preoperative preparation, such as... HbA1c control target Intraoperative precautions, such as strengthening vascular protection for patients with high MVI, and postoperative care plans, such as focusing on monitoring flap blood supply and blood glucose fluctuations, to reduce perioperative risks.
[0025] Furthermore, it also includes a system optimization and learning module, which is used to realize the continuous iterative upgrade of the system, including a clinical outcome collection unit and a model optimization iteration unit; The clinical outcome collection unit is configured to collect and structure-store real-world clinical outcome data related to the patient's breast reconstruction surgery, including at least the actual surgical record, postoperative complications, and long-term follow-up results. The clinical outcome collection unit is a structured storage unit for patients' postoperative real-world data, including actual surgical records, postoperative complications within 30 days, such as flap necrosis and hematoma, and postoperative follow-up results at 6 months or 1 year, such as breast shape satisfaction and functional recovery status. The model optimization iteration unit is connected to the clinical results collection unit and the feature calculation and analysis module. It is configured to periodically or according to triggering conditions use new data collected by the clinical results collection unit to incrementally learn or retrain the pre-trained calculation model in the metabolic vascular injury index calculation module to optimize the calculation accuracy of the continuous numerical risk index. Connected to the clinical outcome collection unit and feature calculation and analysis module, it periodically, such as quarterly, or based on trigger conditions, such as the addition of 100 valid cases, utilizes the newly collected clinical data to analyze... The computational model undergoes incremental learning, updating the model weights using the following formula:
[0026] The loss function is defined as:
[0027] in, The updated model weight vector contains , , ,and , For the course of chronic diseases The weighting coefficients, Blood sugar control level The weighting coefficients, For the first Comorbidity risk factors The weighting coefficients, For the first Comorbidities and the first Interaction effect weights among comorbidities; This is the model weight vector before the update, i.e., the training result of the previous version; The learning rate, ranging from 0.001 to 0.01, controls the step size of weight updates to avoid overfitting or slow convergence. For loss function For weight vector The gradient indicates the direction of weight adjustment; The loss function consists of the MSE loss and the L2 regularization term, and is used to measure the model's prediction error. For the first one sample Predicted value; For the first one sample The true value is inversely calibrated based on the occurrence of postoperative complications; for example, if flap necrosis occurs, then... Increased by 0.1; This refers to the sample size of the newly added clinical data; This is the L2 regularization coefficient, with a value of 0.0001, used to suppress model overfitting caused by excessive weights; Where MSE loss is the mean squared error loss, corresponding to the formula in... Its core function is to quantify the model's... The higher the prediction accuracy of the index, the smaller the MSE loss value indicates that the model's predictions are more accurate. The closer the value is to the true value calibrated after surgery, the higher the prediction accuracy of the model; The L2 regularization term is the weight decay term, corresponding to the formula in... Its core function is to penalize excessively large weight parameters and prevent the model from overfitting. In clinical practice... The training data for the model is limited. If only the MSE loss is optimized, the model may overfit the training data, for example, by over-emphasizing the weight of blood glucose control. This leads to severe bias in the model's predictions on new samples with normal blood sugar but combined hypertension. If a certain weight, such as Too large The value of the L2 regularization term will increase, leading to a higher overall loss. Increase; during model optimization, this weight will be actively reduced, such as from 10 to 0.4 for clinical fit, so that the model can consider all features equally, such as disease course, comorbidities, and interaction terms, to ensure generalization ability on new clinical samples. By continuously optimizing the model weights through incremental learning formulas, The accuracy of computational precision and surgical procedure recommendations improves with the accumulation of clinical data. For example, if new data shows that diabetic patients with nephropathy have more severe vascular damage, then... The weights corresponding to kidney disease will be automatically increased; at the same time, the L2 regularization term avoids excessive weights for a single parameter, ensuring the stability and generalization ability of the model in different clinical scenarios, and solving the problems of model rigidity and inability to adapt to new clinical situations.
[0028] Example 2: like Figure 3 As shown, this embodiment provides a method for recommending breast reconstruction procedures based on multimodal data. The method includes the following steps: S100: Simultaneously acquires patient text, laboratory, imaging, and 3D morphological data through the multimodal data acquisition module, completes preprocessing and structured conversion, and outputs the patient's medical history. Blood sugar control level Comorbidities Original image features and original examination features; The S200 module, along with the feature calculation and analysis module, extracts features from the acquired data. The core module generates the MVI index through the metabolic vascular injury index calculation module, while simultaneously outputting normalized image features. and normalization test characteristics Accurately distinguish patient risks under different control states; The S300 and intelligent decision-making and prediction modules generate multimodal fusion features through a weighted fusion formula. Then, after passing through the probability normalization function Calculate the probability of complications for each surgical procedure Output the initial recommended solution; S400, dynamically updating normalized image features based on intraoperative exploration data. and normalization test characteristics Recalculate multimodal fusion features With probability normalization function And calculate the probability of complications for each surgical procedure. Optimize recommendation schemes in real time; S500 provides visual recommendations, interpretable reports, and management suggestions through its results output and application modules.
[0029] The embodiments disclosed in this invention are preferred embodiments, but are not limited thereto. Those skilled in the art can easily understand the spirit of this invention based on the above embodiments and make different extensions and variations, but as long as they do not depart from the spirit of this invention, they are all within the protection scope of this invention.
Claims
1. A breast reconstruction procedure recommendation system based on multimodal data, characterized in that, The system includes a feature calculation and analysis module, which includes a clinical information structuring unit, a metabolic vascular injury index calculation module, an imaging feature unit, and a test feature unit. The clinical information structured unit is configured to extract and quantify clinical features, including the course of chronic diseases and glycemic control levels, from patient clinical text data. The metabolic vascular injury index calculation module is connected to the information structuring unit and includes an input processing unit, a core calculation unit, and an output generation unit. The input processing unit is configured to receive quantified chronic disease course and blood glucose control level from the clinical information structuring unit, and to receive at least one comorbidity risk factor from the clinical information structuring unit. The core computing unit is configured to perform nonlinear fusion and interaction effect analysis on the received chronic disease course, blood glucose control level and comorbidity risk factors through a pre-trained computing model, and generate a continuous numerical risk index characterizing the degree of metabolic and vascular cumulative damage in patients. The output interface unit is configured to generate and output a continuous numerical risk index characterizing the degree of cumulative damage to the patient's metabolic and vascular systems based on the analysis results of the core computing unit. The image feature unit is configured to extract anatomical and physiological features related to breast reconstruction surgery from medical image data. The test feature analysis unit is configured to process and analyze the patient's laboratory test data.
2. The breast reconstruction procedure recommendation system based on multimodal data according to claim 1, characterized in that, The nonlinear fusion and interaction effect analysis of the core computing unit is achieved through the following formula: ; in, The metabolic vascular damage index has a value range of 0-1. The Sigmoid nonlinear transformation function has the following core expression: ; The course of the disease is chronic; This is a nonlinear normalization function for the disease course; Blood sugar control level; A standardized function for blood glucose control levels; For the first Comorbidity risk factors; A function to identify the comorbidity status; , ,and These are the weighting coefficients for disease duration, glycemic control level, and comorbidities, respectively, satisfying... ; For the first Comorbidities and the first Interaction weights between comorbidities; This is the model calibration bias term, with a value range of [value range missing]. .
3. The breast reconstruction procedure recommendation system based on multimodal data according to claim 1, characterized in that, It also includes a multimodal data acquisition module, which comprises a text data unit, a test data unit, an image data unit, and a 3D morphology analysis unit; The text data unit is used for electronic medical record mining, medical history information extraction, and clinical report parsing; The test data unit is used to process test indicator data such as blood glucose series and HbA1c sequence. The image data unit is used to analyze breast-related image data such as CT / CTA, MRI, and ultrasound. The 3D morphological analysis unit is used to collect and preprocess the patient's 3D body surface morphological data, which includes breast volume, three-dimensional contour symmetry, and body surface topological features.
4. The breast reconstruction procedure recommendation system based on multimodal data according to claim 3, characterized in that, It also includes an intelligent decision-making and prediction module, which comprises a multimodal fusion decision-making unit and an intraoperative dynamic adjustment unit; The multimodal fusion decision unit is connected to the feature calculation and analysis module and is configured to receive a continuous numerical risk index from the metabolic vascular injury index calculation module, anatomical structure features and physiological state features from the image feature unit, and test index features from the test feature unit. Using the continuous numerical risk index as a priori risk guide, the unit performs weighted fusion on the received multimodal features and predicts the probability of various complications for different breast reconstruction procedures based on the fused features. The intraoperative dynamic adjustment unit is configured to receive real-time intraoperative exploration data, trigger the updating of features output by the image feature unit or the verification feature unit based on the exploration data, and notify the multimodal fusion decision unit to re-predict based on the updated features.
5. A breast reconstruction procedure recommendation system based on multimodal data according to claim 4, characterized in that, The weighted fusion of the multimodal fusion decision unit is achieved through the following formula: Multimodal feature weighted fusion formula: ； Formula for predicting the probability of complications: ； in, This is a multimodal fusion feature vector; for The guiding weight of the index; These are the weighting coefficients for image features; This represents the normalized image feature vector; To normalize and test the eigenvectors; For specific surgical procedure numbers; For the first The probability of complications associated with this surgical procedure; For the first Feature weight matrix of the technique; This represents the total number of available breast reconstruction procedures.
6. The breast reconstruction procedure recommendation system based on multimodal data according to claim 4, characterized in that, It also includes a results output and application module, which includes an intraoperative recommendation visualization unit, an interpretable report generation unit, and a clinical management key points unit; The intraoperative recommendation visualization unit is configured to receive and display updated surgical risk assessments and adjustment suggestions in real time during the operation. The interpretable report generation unit is configured to generate an interpretable report containing a decision logic chain, which is used to clarify the correlation between the continuously numerical risk index output by the metabolic vascular injury index calculation module and the fact that a specific surgical procedure is marked as high-risk or recommended. The clinical management key points unit is configured to automatically match and generate a personalized list of perioperative management recommendations based on a continuous numerical risk index and the selected surgical procedure.
7. A breast reconstruction procedure recommendation system based on multimodal data according to claim 4, characterized in that, It also includes a system optimization and learning module, which comprises a clinical outcome collection unit and a model optimization iteration unit; The clinical outcome collection unit is configured to collect and structure-store real-world clinical outcome data related to the patient's breast reconstruction surgery, including at least the actual surgical record, postoperative complications, and long-term follow-up results. The model optimization iteration unit is connected to the clinical results collection unit and the feature calculation and analysis module. It is configured to periodically or according to triggering conditions use new data collected by the clinical results collection unit to incrementally learn or retrain the pre-trained calculation model in the metabolic vascular injury index calculation module to optimize the calculation accuracy of the continuous numerical risk index.
8. A breast reconstruction procedure recommendation system based on multimodal data according to claim 7, characterized in that, The incremental learning of the model optimization iteration unit updates the model weights using the following formula: ； The loss function is defined as: ； in, This is the updated model weight vector; For the course of chronic diseases The weighting coefficients, Blood sugar control level The weighting coefficients, For the first Comorbidity risk factors The weighting coefficients, For the first Comorbidities and the first Interaction effect weights among comorbidities; The model weight vector before the update; The learning rate; For loss function For weight vector The gradient; The loss function; For the first one sample Predicted value; For the first one sample The actual value; is the L2 regularization coefficient.
9. A method for recommending breast reconstruction procedures based on multimodal data, applicable to the breast reconstruction procedure recommendation system based on multimodal data as described in any one of claims 1-8, characterized in that, The method includes the following steps: S100: Simultaneously acquires patient text, laboratory, imaging, and 3D morphological data through the multimodal data acquisition module, completes preprocessing and structured conversion, and outputs the patient's medical history. Blood sugar control level Comorbidities Original image features and original examination features; The S200 module, along with the feature calculation and analysis module, extracts features from the acquired data. The core module generates the MVI index through the metabolic vascular injury index calculation module, while simultaneously outputting normalized image features. and normalization test characteristics Accurately distinguish patient risks under different control states; The S300 and intelligent decision-making and prediction modules generate multimodal fusion features through a weighted fusion formula. Then, after passing through the probability normalization function Calculate the probability of complications for each surgical procedure Output the initial recommended solution; S400, dynamically updating normalized image features based on intraoperative exploration data. and normalization test characteristics Recalculate multimodal fusion features With probability normalization function And calculate the probability of complications for each surgical procedure. Optimize recommendation schemes in real time. S500 provides visual recommendations, interpretable reports, and management suggestions through results output and application modules; S600 collects postoperative clinical results and updates model weights through incremental learning formulas to achieve continuous optimization of system performance.