BMI driving type 3D organ obesity visualization system based on multi-modal fusion

Abstract

The invention discloses a BMI-driven 3D organ obesity visualization system based on multi-modal fusion, and relates to the technical field of medical science, the system comprises a BMI-driven prediction model, the BMI-driven prediction model is used for establishing a quantitative mapping model of a BMI index and three-dimensional organ fat deposition, and the quantitative mapping model is used for establishing a BMI index and three-dimensional organ fat deposition. The BMI driving type prediction model comprises a multi-modal data fusion unit, a model mapping unit and a residual compensation unit. By installing the BMI driving type prediction model, precise nonlinear mapping of the BMI index and organ fat deposition is achieved, the problem that traditional single-mode prediction is large in deviation is solved, and organ fat volume prediction precision and individual difference adaptability are improved.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a BMI-driven 3D organ obesity visualization system based on multimodal fusion. Background Art

[0002] The global obesity problem is becoming increasingly serious. As of 2023, data from the World Health Organization shows that the number of obese people in the world has exceeded 650 million, and the obesity rate of Chinese adult residents is as high as 50.7%. The surge in global obesity rates has led to an increased burden of diseases such as metabolic syndrome. The traditional BMI indicator has the defect of insufficient assessment of organ-specific fat distribution.

[0003] Patent CN112168211B discloses a method and system for measuring fat thickness and muscle thickness in abdominal ultrasound images. The above patent realizes a simple and efficient calculation of fat layer thickness and muscle layer thickness, and can also obtain a clear and intuitive visual structure diagram showing the abdominal tissue structure.

[0004] The above patent includes collecting abdominal ultrasound images, selecting a part as the image to be processed, scaling to obtain the image to be segmented, segmenting the tissue structure of the image to be segmented to obtain a first tissue structure image, scaling the first tissue structure image to obtain a second tissue image that is the same size as the image to be processed, and using the second fat layer image and the second muscle layer image in the second tissue structure image to calculate the fat layer thickness and the muscle layer thickness; it also includes pseudo-coloring the image to be processed and the second tissue structure image to obtain a visual structure diagram; the present invention can simply and efficiently calculate the fat layer thickness and the muscle layer thickness, and can also obtain a clear and intuitive visual structure diagram showing the abdominal tissue structure, but the above patent has shortcomings in constructing fat visualization through multimodal fusion.

[0005] To this end, this application proposes a BMI-driven 3D organ obesity visualization system based on multimodal fusion that can realize multimodal fusion to construct fat visualization. Summary of the Invention

[0006] The purpose of the present invention is to provide a BMI-driven 3D organ obesity visualization system based on multimodal fusion to solve the technical problems raised in the above background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solutions: a BMI-driven 3D organ obesity visualization system based on multimodal fusion, comprising a BMI-driven prediction model for establishing a quantitative mapping model between BMI index and three-dimensional organ fat deposition;

[0008] The BMI-driven prediction model includes: a multimodal data fusion unit, a model mapping unit and a residual compensation unit;

[0009] The multimodal data fusion unit is configured with a multimodal data fusion algorithm, which performs spatiotemporal alignment of the multimodal data acquisition module data through transfer learning, dynamically allocates the fusion weight of each modality data based on the attention mechanism, and eliminates motion artifacts based on the wavelet transform through the denoising function;

[0010] The dynamic mapping network unit integrates a BMI-organ fat three-dimensional mapping model based on a hybrid architecture of Transformer and graph convolutional neural network to establish a nonlinear mapping relationship between BMI index and organ fat volume;

[0011] The residual compensation unit uses a deep residual network to perform nonlinear correction on the output of the dynamic mapping network unit to compensate for the prediction deviation caused by individual differences.

[0012] Preferably, the BMI-driven prediction model is signal-connected to a multimodal data acquisition module, and the multimodal data acquisition module is used to collect various human body data for the BMI-driven prediction model;

[0013] The multimodal data acquisition module includes: bioelectrical impedance acquisition unit, ultrasonic elastography unit and near-infrared spectroscopy unit;

[0014] The bioelectrical impedance acquisition unit is connected to the bioimpedance spectrometer to measure the body fat distribution characteristics through multi-band current excitation and extract the phase angle parameters;

[0015] The ultrasound elastography unit is connected to a high-frequency ultrasound elastography instrument to detect the elastic parameters of organ fat through shear wave velocity and generate an organ tissue hardness distribution map;

[0016] The near-infrared spectroscopy unit is connected to the near-infrared spectroscopy sensor, uses a dual-wavelength light source to detect the oxygen metabolism parameters of subcutaneous fat, and separates fat signals at different subcutaneous depths through a depth resolution algorithm.

[0017] Preferably, the BMI-driven prediction model is signal-connected to a three-dimensional visualization module, and the three-dimensional visualization module is used to implement real-time dynamic rendering of the BMI-driven prediction model;

[0018] The 3D visualization module includes: physical rendering unit, heat map mapping unit and dynamic prediction unit;

[0019] The physical rendering unit is based on a GPU-accelerated parallel rendering pipeline and uses a physical lighting model to perform 3D reconstruction of organ fat deposition;

[0020] The heat map mapping unit maps the fat density gradient into a color gradient using the HSL color space conversion algorithm and superimposes the metabolic risk level indicator;

[0021] The dynamic prediction unit integrates an improved convolutional long short-term memory network to simulate the spatiotemporal evolution of organ fat deposition and generate prediction visualization results for future time nodes.

[0022] Preferably, the system is also designed with a health management APP, which includes a closed-loop management module, realizes full-link management of monitoring-early warning-intervention through multimodal data collaboration, and supports cross-platform operation of HarmonyOS, iOS and Android;

[0023] The closed-loop management module includes: wearable device access unit, intelligent warning center and dynamic intervention engine;

[0024] The wearable device access unit integrates a multi-protocol adaptation layer, supports heterogeneous device data fusion such as Bluetooth body fat scales, smart bracelets, and NIRS sensors, and achieves synchronous collection of multi-source physiological parameters through a timestamp alignment algorithm.

[0025] The intelligent early warning center is connected to the dynamic prediction unit, triggering a graded early warning strategy when the organ fat deposition rate exceeds a threshold;

[0026] The dynamic intervention engine includes an AI nutritionist, which adjusts nutritional intake and exercise plans in real time based on a reinforcement learning framework and provides users with medication reminders.

[0027] Preferably, the system is also designed with a federated learning framework module for multi-center data joint modeling;

[0028] The multi-center federated learning framework includes: local model update unit, global aggregation unit and privacy protection unit;

[0029] The local model training unit trains the sub-model of the BMI-driven prediction model locally in each medical institution;

[0030] The global aggregation unit uses differential privacy technology to perform weighted averaging on the parameters of the sub-models to generate a global shared model;

[0031] The privacy protection unit uses a homomorphic encryption algorithm to ensure the invisibility of the original data during transmission.

[0032] Preferably, the system is also designed with a surgical assistance module for achieving precise metabolic surgical support;

[0033] The surgical assistance module includes: fat blood vessel 3D reconstruction unit, virtual resection planning unit and intraoperative AR navigation unit;

[0034] The fat vascular 3D reconstruction unit generates a topological map of the vascular-fat spatial relationship by distinguishing the fat infiltration characteristics around the portal vein and hepatic vein;

[0035] The virtual resection simulation unit contains a biomechanical simulation model, calculates organ deformation parameters through finite element analysis, and drives the 3D visualization module in real time to display the predicted image results;

[0036] The intraoperative AR navigation unit uses Hololens2 to superimpose organ fat heat maps with the real surgical field of view, and integrates depth sensors to achieve millimeter-level alignment of virtual and real scenes, allowing real-time navigation to avoid dangerous triangles.

[0037] Preferably, the multimodal data fusion algorithm configured by the multimodal data fusion unit is as follows:

[0038]

[0039] Among them, FatMap represents the fused fat distribution heat map, which is the final output of multimodal fusion result and is used to drive 3D visualization; i is the dynamic weight of the i-th modal data, calculated through the attention mechanism; D i is the original data of the i-th mode, that is, the data transmitted by the multimodal data acquisition module; Normalize() is a data normalization function used to scale data of different dimensions to the [0,1] interval; λ i is the noise reduction coefficient, which is related to the wavelet transform level; Noise() is the noise estimation function, which is based on the energy calculation of the high-frequency component of wavelet decomposition.

[0040] Preferably, the BMI-organ fat three-dimensional mapping model integrated by the dynamic mapping network unit is as follows:

[0041] V organ =α·BMI β +γ·Age+δ·Sex+∈·WHR+NN residual (X)

[0042] V organ It is a quantitative indicator of three-dimensional organ fat deposition, which is used to represent the fat volume or density of a specific organ; α is the BMI basic coefficient, which is used to control the linear effect of BMI on organ fat; β is the BMI nonlinear index, which is used to control the nonlinear effect of BMI on organ fat; γ is the age coefficient, which is used to control the effect of age on organ fat; δ is the sex coefficient, which is used to control the effect of sex on organ fat; ε is the waist-to-hip ratio coefficient, which is used to control the effect of waist-to-hip ratio on organ fat; NN residual (X) is a residual neural network with input X = [HbA1c, insulin, CT value] to compensate for nonlinear effects not captured by the model.

[0043] Preferably, the management method comprises the following steps:

[0044] S1. Synchronous multimodal data acquisition: Bioimpedance spectrometer, ultrasound elastography, and near-infrared spectroscopy sensors are used to synchronously acquire the subject's biophysical characteristics.

[0045] S2. Cross-modal feature fusion: Utilize the transfer learning framework to align the feature representations of data from different modalities and dynamically fuse multi-source data through the attention mechanism;

[0046] S3. Organ fat volume prediction: BMI and fusion features are input into the quantitative mapping model to calculate the three-dimensional fat distribution parameters of each organ;

[0047] S4. Dynamic visualization: Generate organ fat heat maps based on a physical rendering engine and overlay metabolic risk warning information.

[0048] Preferably, the management method further comprises the following steps:

[0049] S11. Add motion artifact elimination processing during the data acquisition stage and use wavelet transform algorithm to filter out noise interference in bioimpedance measurement;

[0050] S21. Implement federated learning optimization during the feature fusion process and regularly aggregate the model parameter updates of each medical institution;

[0051] S31. Enable AR interaction function during visualization and achieve three-dimensional spatial annotation of fat deposition areas through head-mounted display devices;

[0052] S41. Integrate a real-time feedback mechanism into the health management closed loop and dynamically adjust intervention strategy parameters based on user compliance.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. By installing a BMI-driven prediction model, this invention achieves accurate nonlinear mapping between BMI index and organ fat deposition, solving the problem of large deviation in traditional single-modality prediction and improving the accuracy of organ fat volume prediction and adaptability to individual differences;

[0055] 2. By installing a multimodal data acquisition module, this invention overcomes the technical bottleneck of a single detection method being unable to fully characterize fat distribution characteristics, and improves the comprehensive detection capabilities of subcutaneous fat, organ fat, and metabolic parameters;

[0056] 3. By installing a three-dimensional visualization module, this invention solves the problem that static visualization cannot reflect the spatiotemporal evolution of fat, and realizes the visual dynamic simulation of the organ fat deposition process and the prediction of future trends;

[0057] 4. The present invention solves the contradiction between health management data silos and privacy protection by installing a closed-loop management module and a federated learning framework module, and realizes the dynamic optimization of cross-platform health intervention strategies and the secure joint modeling of multi-center data. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0059] Figure 2 Schematic diagram of the multimodal data fusion unit process of the present invention;

[0060] Figure 3 This is a schematic structural diagram of the surgical auxiliary module of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] In the description of the present invention, it should be noted that the terms "upper," "lower," "inner," "outer," "front end," "rear end," "both ends," "one end," "the other end," and the like, indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limiting the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0063] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "provided with," "connected," etc., should be understood in a broad sense. For example, "connected" may refer to a fixed connection, a detachable connection, or an integral connection; it may refer to a mechanical connection or an electrical connection; it may refer to a direct connection or an indirect connection through an intermediate medium; it may refer to internal communication between two components. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0064] See also Figure 1 , an embodiment provided by the present invention: a BMI-driven 3D organ obesity visualization system based on multimodal fusion, comprising a BMI-driven prediction model, wherein the BMI-driven prediction model is used to establish a quantitative mapping model between BMI index and three-dimensional organ fat deposition;

[0065] The BMI-driven prediction model includes: a multimodal data fusion unit, a model mapping unit and a residual compensation unit;

[0066] The multimodal data fusion unit is configured with a multimodal data fusion algorithm, which performs spatiotemporal alignment of the multimodal data acquisition module data through transfer learning, dynamically allocates the fusion weight of each modality data based on the attention mechanism, and eliminates motion artifacts based on the wavelet transform through the denoising function;

[0067] The dynamic mapping network unit integrates a BMI-organ fat three-dimensional mapping model based on a hybrid architecture of Transformer and graph convolutional neural network to establish a nonlinear mapping relationship between BMI index and organ fat volume;

[0068] The residual compensation unit uses a deep residual network to perform nonlinear correction on the output of the dynamic mapping network unit to compensate for the prediction deviation caused by individual differences;

[0069] The BMI-driven prediction model is signal-connected to a multimodal data acquisition module, which is used to collect various human body data for the BMI-driven prediction model;

[0070] The multimodal data acquisition module includes: bioelectrical impedance acquisition unit, ultrasonic elastography unit and near-infrared spectroscopy unit;

[0071] The bioelectrical impedance acquisition unit is connected to the bioimpedance spectrometer to measure the body fat distribution characteristics through multi-band current excitation and extract the phase angle parameters;

[0072] The ultrasound elastography unit is connected to a high-frequency ultrasound elastography instrument to detect the elastic parameters of organ fat through shear wave velocity and generate an organ tissue hardness distribution map;

[0073] The near-infrared spectroscopy unit is connected to the near-infrared spectroscopy sensor, which uses a dual-wavelength light source to detect the oxygen metabolism parameters of subcutaneous fat and separates fat signals at different subcutaneous depths through a depth resolution algorithm;

[0074] The BMI-driven prediction model is signal-connected to a three-dimensional visualization module, which is used to implement real-time dynamic rendering of the BMI-driven prediction model;

[0075] The 3D visualization module includes: physical rendering unit, heat map mapping unit and dynamic prediction unit;

[0076] The physical rendering unit is based on a GPU-accelerated parallel rendering pipeline and uses a physical lighting model to perform 3D reconstruction of organ fat deposition;

[0077] The heat map mapping unit maps the fat density gradient into a color gradient using the HSL color space conversion algorithm and superimposes the metabolic risk level indicator;

[0078] The dynamic prediction unit integrates an improved convolutional long short-term memory network to simulate the spatiotemporal evolution of organ fat deposition and generate prediction visualization results for future time nodes;

[0079] Furthermore, the system first synchronously acquires human body data through a bioelectrical impedance acquisition unit, an ultrasound elastography unit, and a near-infrared spectroscopy unit. The bioelectrical impedance acquisition unit measures whole-body fat distribution characteristics through multi-band current excitation, extracts phase angle parameters, and combines them with the organ tissue stiffness distribution map generated by the ultrasound elastography unit and the subcutaneous fat oxygen metabolism parameters separated by the near-infrared spectroscopy unit to complete the original multimodal data input. The multimodal data fusion unit aligns these data in time and space through transfer learning and dynamically assigns weights to each modality using an attention mechanism. For example, in the liver fat assessment scenario, ultrasound elastography data may be given a higher weight because it directly reflects tissue stiffness. Simultaneously, a wavelet transform algorithm denoises motion artifacts in the bioelectrical impedance data to ensure the accuracy of the phase angle parameters. The fused features are then input to the dynamic mapping network unit, which establishes a nonlinear relationship between BMI and organ fat volume through a hybrid architecture of a Transformer and a graph convolutional network. For example, when BMI = 28.4, the model calculates liver fat volume based on the nonlinear coefficient β and introduces individualized parameters through the residual compensation unit to correct prediction deviations.

[0080] In addition, after the 3D visualization module receives the prediction results, the physical rendering unit uses a GPU-accelerated physical lighting model to reconstruct the 3D structure of the liver. The heat map mapping unit converts the fat density gradient into a color gradient in the HSL color space. For example, the S6 segment of the right lobe of the liver shows a red high-risk area. The dynamic prediction unit simulates the spatiotemporal evolution of future fat deposition through a convolutional long short-term memory network. The final generated 3D report is presented in the form of a heat map around the portal vein in a VR environment and can be output to the outpatient system within 5 minutes.

[0081] See also Figure 1 The present invention provides an embodiment of a BMI-driven 3D organ obesity visualization system based on multimodal fusion. The system is also designed with a health management app. The health management app includes a closed-loop management module, which realizes full-link management of monitoring, early warning, and intervention through the collaboration of multimodal data, and supports cross-platform operation of HarmonyOS, iOS, and Android.

[0082] The closed-loop management module includes: wearable device access unit, intelligent warning center and dynamic intervention engine;

[0083] The wearable device access unit integrates a multi-protocol adaptation layer, supports heterogeneous device data fusion such as Bluetooth body fat scales, smart bracelets, and NIRS sensors, and achieves synchronous collection of multi-source physiological parameters through a timestamp alignment algorithm.

[0084] The intelligent early warning center is connected to the dynamic prediction unit, triggering a graded early warning strategy when the organ fat deposition rate exceeds a threshold;

[0085] The dynamic intervention engine includes an AI nutritionist that adjusts nutrition intake and exercise plans in real time based on a reinforcement learning framework, and provides medication reminders to users;

[0086] The system is also designed with a federated learning framework module for multi-center data joint modeling;

[0087] The multi-center federated learning framework includes: local model update unit, global aggregation unit and privacy protection unit;

[0088] The local model training unit trains the sub-model of the BMI-driven prediction model locally in each medical institution;

[0089] The global aggregation unit uses differential privacy technology to perform weighted averaging on the parameters of the sub-models to generate a global shared model;

[0090] The privacy protection unit uses a homomorphic encryption algorithm to ensure the invisibility of original data during transmission;

[0091] Furthermore, in the closed-loop health management process, the wearable device access unit collects real-time physiological data, such as measurements from body composition meters, through heterogeneous devices such as Bluetooth body fat scales and smart bracelets. A timestamp alignment algorithm ensures the synchronization of multi-source data, such as body fat percentage, heart rate, and blood oxygen. When the intelligent early warning center detects abnormal organ fat deposition rates, such as an annual liver fat growth rate of >5%, a graded early warning strategy is triggered: high-risk patients are automatically prescribed metabolomics testing, medium-risk patients initiate mobile AI nutritionist intervention, and low-risk patients receive annual follow-up reminders. The federated learning framework supports multi-center data joint modeling. For example, the local model training unit of a tertiary hospital updates the sub-model parameters of a BMI-driven prediction model based on its clinical data. The privacy protection unit uploads the parameters to the global aggregation unit through homomorphic encryption. The global model uses differential privacy technology to perform a weighted average of the sub-model parameters of 10 medical institutions to generate a shared model, which is then distributed to each institution. During this process, the original patient data is always retained locally, and only the model parameters are exchanged, which not only achieves data privacy protection but also improves model generalization capabilities.

[0092] See also Figure 3The present invention provides an embodiment of a BMI-driven 3D organ obesity visualization system based on multimodal fusion, wherein the system is also designed with a surgical assistance module for achieving precise metabolic surgical support;

[0093] The surgical assistance module includes: fat blood vessel 3D reconstruction unit, virtual resection planning unit and intraoperative AR navigation unit;

[0094] The fat vascular 3D reconstruction unit generates a topological map of the vascular-fat spatial relationship by distinguishing the fat infiltration characteristics around the portal vein and hepatic vein;

[0095] The virtual resection simulation unit contains a biomechanical simulation model, calculates organ deformation parameters through finite element analysis, and drives the 3D visualization module in real time to display the predicted image results;

[0096] The intraoperative AR navigation unit uses Hololens2 to overlay organ fat thermal maps with the real surgical field of view, and integrates a depth sensor to achieve millimeter-level alignment of virtual and real scenes, allowing real-time navigation to avoid dangerous triangles.

[0097] Furthermore, during the preoperative planning stage, the fat vascular 3D reconstruction unit uses multimodal data fusion to distinguish the spatial characteristics of fat deposits around the portal and hepatic veins. For example, based on shear wave velocity data from ultrasound elastography and oxygen metabolism parameters from near-infrared spectroscopy, the system identifies high-risk areas with fat infiltration thickness greater than 5 mm around the portal vein, while fat deposits around the hepatic veins exhibit lower density. The 3D reconstruction algorithm uses topological mapping technology to convert the geometric relationship between vascular branches and areas of fat infiltration into a spatial atlas: the tertiary branches of the portal vein are annotated with green wireframes, surrounded by a red heat map representing the fat infiltration gradient, while a blue translucent layer identifies the hepatic vein drainage area. The virtual resection planning unit integrates a biomechanical simulation model and simulates the impact of different surgical plans on organ function based on finite element analysis. For example, when planning to resect 30% of the right lobe of the liver, the system calculates the stress distribution of the remaining liver tissue. By inputting liver elastic modulus, vascular tension parameters, and hemodynamic data, the system predicts the risk of postoperative portal hypertension and dynamically displays the organ deformation trend after resection through a 3D visualization module.

[0098] In addition, the intraoperative AR navigation unit uses Hololens2 to achieve millimeter-level registration of virtual and real scenes. The preoperative planned heat map is aligned with the patient's real anatomical structure through the following steps: the depth sensor collects liver surface point cloud data, the feature matching algorithm identifies the portal vein bifurcation and gallbladder fossa anatomical landmarks, and the spatial transformation matrix maps the virtual model coordinates to the real organ coordinate system. In the surgeon's field of view, the danger triangle area around the portal vein-bile duct complex is highlighted as a flashing red frame, and the fat deposition heat map is superimposed on the liver surface as a translucent layer. When the ultrasonic knife tip is less than 2mm away from the danger zone, the system triggers an alarm: the AR interface displays a red warning line, and the ultrasonic lipolysis equipment power is automatically reduced from the standard gear to the safe gear. The drug-device linkage module adjusts parameters according to the real-time fat distribution: if the heat map shows that the local fat density is greater than 6%, the system increases the lipolysis pulse frequency to 5Hz and extends the action time to 500ms to ensure maximum efficiency in fat cell fragmentation.

[0099] During the postoperative verification phase, the system compares and analyzes the real-time data collected during the operation with the preoperative prediction results. For example, if the shear wave velocity of the right lobe of the liver drops to 1.8m / s after actual resection, and the preoperative prediction is 1.7m / s, the residual compensation unit automatically updates the model parameters and optimizes the compensation rate prediction algorithm for subsequent operations. At the same time, the blockchain health file records key surgical data, generates traceable NFT surgical reports, and supports cross-institutional diagnosis and treatment collaboration.

[0100] See also Figure 1 , an embodiment provided by the present invention: a BMI-driven 3D organ obesity visualization system based on multimodal fusion, comprising a BMI-driven prediction model, wherein the BMI-driven prediction model is used to establish a quantitative mapping model between BMI index and three-dimensional organ fat deposition;

[0101] The BMI-driven prediction model includes: a multimodal data fusion unit, a model mapping unit and a residual compensation unit;

[0102] The multimodal data fusion unit is configured with a multimodal data fusion algorithm, which performs spatiotemporal alignment of the multimodal data acquisition module data through transfer learning, dynamically allocates the fusion weight of each modality data based on the attention mechanism, and eliminates motion artifacts based on the wavelet transform through the denoising function;

[0103] The dynamic mapping network unit integrates a BMI-organ fat three-dimensional mapping model based on a hybrid architecture of Transformer and graph convolutional neural network to establish a nonlinear mapping relationship between BMI index and organ fat volume;

[0104] The residual compensation unit uses a deep residual network to perform nonlinear correction on the output of the dynamic mapping network unit to compensate for the prediction deviation caused by individual differences;

[0105] The system is also designed with a health management app, which includes a closed-loop management module. It uses multimodal data collaboration to achieve full-link management of monitoring, early warning, and intervention, and supports cross-platform operation on HarmonyOS, iOS, and Android.

[0106] The closed-loop management module includes: wearable device access unit, intelligent warning center and dynamic intervention engine;

[0107] The wearable device access unit integrates a multi-protocol adaptation layer, supports heterogeneous device data fusion such as Bluetooth body fat scales, smart bracelets, and NIRS sensors, and achieves synchronous collection of multi-source physiological parameters through a timestamp alignment algorithm.

[0108] The intelligent early warning center is connected to the dynamic prediction unit, triggering a graded early warning strategy when the organ fat deposition rate exceeds a threshold;

[0109] The dynamic intervention engine includes an AI nutritionist that adjusts nutrition intake and exercise plans in real time based on a reinforcement learning framework, and provides medication reminders to users;

[0110] Furthermore, the wearable device access unit synchronizes the impedance data of the body fat scale, the heart rate variability parameters of the smart bracelet, and the subcutaneous fat oxygen saturation of the NIRS sensor through the Bluetooth 5.0 protocol. The sliding time window algorithm is used to align the sampling timestamps of different devices. The intelligent early warning center detects that the liver fat deposition rate exceeds 1.5cm for three consecutive days. 3 / day, a three-level warning mechanism is activated: the first-level warning pushes health reminders through the APP, the second-level warning triggers SMS notifications, and the third-level warning will automatically call the preset emergency contact number and pop up a red full-screen warning on the health management APP interface. The AI ​​nutritionist in the dynamic intervention engine is based on the reinforcement learning framework to analyze the user's diet records and exercise consumption for the past 72 hours in real time: when an abnormal fat deposition rate is detected, the daily carbohydrate intake limit is immediately reduced by 20%, and a high-intensity interval training program is recommended. If the user fails to achieve the exercise goal for two consecutive days, the compensatory mechanism is activated, and the protein intake ratio is automatically increased to 35% the next day. The medication reminder module will take into account the severity of fatty liver and use vibration reminders to take bicyclol tablets 30 minutes before meals. The medication action is verified by the camera. The user's compliance data will be fed back to the residual compensation unit to optimize the parameters of the individualized prediction model. For example, for users who often stay up late, the cortisol hormone influencing factor is increased.

[0111] See also Figure 1 and Figure 2, an embodiment provided by the present invention: a BMI-driven 3D organ obesity visualization system based on multimodal fusion, comprising a BMI-driven prediction model, wherein the BMI-driven prediction model is used to establish a quantitative mapping model between BMI index and three-dimensional organ fat deposition;

[0112] The BMI-driven prediction model includes: a multimodal data fusion unit, a model mapping unit and a residual compensation unit;

[0113] The multimodal data fusion unit is configured with a multimodal data fusion algorithm, which performs spatiotemporal alignment of the multimodal data acquisition module data through transfer learning, dynamically allocates the fusion weight of each modality data based on the attention mechanism, and eliminates motion artifacts based on the wavelet transform through the denoising function;

[0114] The dynamic mapping network unit integrates a BMI-organ fat three-dimensional mapping model based on a hybrid architecture of Transformer and graph convolutional neural network to establish a nonlinear mapping relationship between BMI index and organ fat volume;

[0115] The residual compensation unit uses a deep residual network to perform nonlinear correction on the output of the dynamic mapping network unit to compensate for the prediction deviation caused by individual differences;

[0116] The multimodal data fusion algorithm configured by the multimodal data fusion unit is as follows:

[0117]

[0118] Among them, FatMap represents the fused fat distribution heat map, which is the final output of multimodal fusion result and is used to drive 3D visualization; i is the dynamic weight of the i-th modal data, calculated through the attention mechanism; D i is the original data of the i-th mode, that is, the data transmitted by the multimodal data acquisition module; Normalize() is a data normalization function used to scale data of different dimensions to the [0,1] interval; λ i is the noise reduction coefficient, which is related to the wavelet transform level; Noise() is the noise estimation function, which is based on the energy calculation of the high-frequency component of wavelet decomposition.

[0119] The BMI-organ fat three-dimensional mapping model integrated by the dynamic mapping network unit is as follows:

[0120] V organ =α·BMI β +γ·Age+δ·Sex+∈·WHR+NN residual (X)

[0121] V organIt is a quantitative indicator of three-dimensional organ fat deposition, which is used to represent the fat volume or density of a specific organ; α is the BMI basic coefficient, which is used to control the linear effect of BMI on organ fat; β is the BMI nonlinear index, which is used to control the nonlinear effect of BMI on organ fat; γ is the age coefficient, which is used to control the effect of age on organ fat; δ is the sex coefficient, which is used to control the effect of sex on organ fat; ε is the waist-to-hip ratio coefficient, which is used to control the effect of waist-to-hip ratio on organ fat; NN residual (X) is the residual neural network with input X = [HbA1c, insulin, CT value], which compensates for the nonlinear effects not captured by the model;

[0122] Furthermore, the multimodal data fusion unit receives the phase angle matrix provided by the bioelectrical impedance acquisition unit, the shear wave velocity tensor output by the ultrasound elastic imaging unit, and the oxygen metabolism depth distribution map generated by the near-infrared spectroscopy unit. In the spatiotemporal alignment stage, the transfer learning framework aligns the spatial coordinate system of the ultrasound data with the surface projection grid of the near-infrared data, and eliminates the coordinate offset caused by body position differences through affine transformation. The attention mechanism dynamically calculates the confidence of each modality during the fusion process: when the subject's movement causes the near-infrared signal signal-to-noise ratio to drop below 3dB, its weight coefficient w_i automatically decays, and at the same time, the weight of the ultrasound elasticity data is increased. The denoising function performs threshold processing on the third-layer detail coefficients of the wavelet decomposition, effectively filtering out the 0.1-0.3 dB caused by respiratory movement in the bioimpedance measurement. Hz frequency band noise. In the dynamic mapping network unit, the Transformer module encodes morphological parameters such as BMI index and waist-to-hip ratio into a 128-dimensional feature vector, and captures the nonlinear correlation between BMI mutation points and liver fat volume through a multi-head attention mechanism. The graph convolutional neural network constructs an organ connection map based on the human anatomical atlas. When the residual compensation unit receives the correction value output by NNresidual(X), it will give priority to abnormal fat deposition areas with CT values ​​>10HU. For example, for patients with congenital intrahepatic bile duct dilatation, an additional volume compensation coefficient is added to the left lateral lobe of the liver. Finally, the three-dimensional visualization module superimposes the fused FatMap with anatomical landmarks. When the visceral fat area exceeds the limit, an orange risk mark is automatically marked at the diaphragm attachment.

[0123] Working Principle: First, the system synchronously acquires multi-dimensional biometric features of the human body through a multimodal data acquisition module. The bioelectrical impedance acquisition unit is connected to a multi-band impedance spectrometer to measure the whole-body phase angle within the 50kHz-1MHz frequency band and analyze the fat and water distribution characteristics. The ultrasound elastography unit integrates a shear wave velocity detection algorithm to quantify the fat infiltration hardness of target organs such as the liver with an accuracy of 2.3m / s. The near-infrared spectroscopy unit is equipped with a dual-wavelength light source and uses a depth resolution algorithm to separate oxygen metabolism signals at a depth of 3-8mm under the skin. After these heterogeneous data are aligned by time stamps, cross-domain feature extraction is performed through the multimodal data fusion unit: respiratory motion artifacts in the bioelectrical impedance are filtered out based on wavelet transform, and the attention mechanism is used to dynamically allocate the fusion ratio of ultrasound elastography data and near-infrared spectroscopy data to generate a spatiotemporally aligned fat distribution feature matrix.

[0124] Next, the BMI-driven prediction model performs in-depth analysis of the fused data. The dynamic mapping network unit, based on the Transformer architecture, expands the scalar input of BMI = 28.4 into a three-dimensional vector. Combined with a graph convolutional neural network, it captures the topological relationships of organ anatomical structures. For example, the model uses 18 layers of cross-attention heads to establish a nonlinear mapping between BMI and fat volume in the S6 segment of the right lobe of the liver. Simultaneously, the residual compensation unit integrates individualized parameters such as HbA1c and CT values ​​to correct bias in the prediction results. Throughout this process, the federated learning framework regularly aggregates gradient updates from the multi-center model, protecting patient data through differential privacy and continuously optimizing the prediction accuracy of the global model.

[0125] Finally, the 3D visualization module and the surgical assistance module collaborate to output clinical application results. The physical rendering unit, based on a GPU-accelerated parallel rendering pipeline, converts the predicted fat volume into a red heat map around the portal vein. The dynamic prediction unit simulates the evolution of fat deposition over the next six months using an improved convolutional long short-term memory network. For example, the fat coverage of the left hepatic lobe increases from 15% to 22%. In surgical scenarios, the virtual resection planning unit uses finite element analysis to calculate the stress distribution after a 30% right hepatic resection. The intraoperative AR navigation unit uses the HoloLens 2's depth sensor to superimpose the danger triangle onto the real surgical field as a flashing red frame. When the ultrasonic scalpel tip is less than 2mm from the high-risk area, the system automatically reduces power to 30W and triggers an alert. Simultaneously, the health management app dynamically adjusts user intervention plans using a reinforcement learning algorithm. For example, when the predicted fat deposition rate exceeds 1.5% / month, the AI ​​nutritionist automatically reduces daily carbohydrate intake from 150g to 120g and generates a traceable NFT report in sync with the blockchain health record, completing the complete closed loop from data perception to precise intervention.

[0126] Multimodal Data Acquisition Experiment Report

[0127] Experimental purpose: To verify the accuracy and reliability of data collected by the bioelectrical impedance acquisition unit, ultrasound elastography unit, and near-infrared spectroscopy unit, as well as their correlation with obesity-related indicators, and to provide data support for the study of the relationship between BMI and organ fat deposition and the optimization of the visualization system.

[0128] Experimental methods

[0129] Experimental subjects: 300 volunteers of different ages, genders, and BMI ranges were selected, with normal weight (BMI 18.5-23.9 kg / m 2 ) 100 people, overweight (BMI 24-27.9kg / m 2 ) 100 people, obese (BMI ≥ 28 kg / m 2 ) 100 people. Those with severe liver, kidney or other organ diseases and those with a recent history of major surgery were excluded.

[0130] Experimental equipment: The bioelectrical impedance acquisition unit uses a high-precision bioimpedance spectrometer that can measure in the 50kHz-1MHz frequency range; the ultrasonic elastography unit uses a professional high-frequency ultrasonic elastography instrument with a shear wave velocity detection accuracy of 0.01m / s; the near-infrared spectroscopy unit is connected to a near-infrared spectroscopy sensor, using a dual-wavelength light source and depth resolution algorithm.

[0131] Experimental Procedure: In a quiet, constant temperature (25°C ± 1°C) environment, volunteers fasted for 8-12 hours. Basic information was recorded before measurement to ensure relaxation. For bioelectrical impedance collection, electrodes were placed in standard positions, and three measurements were taken in each frequency band, averaging the results. For ultrasound elastography, probes were selected based on the organ being examined, and after applying coupling agent, measurements were taken to generate a hardness distribution map. The near-infrared spectroscopy sensor was placed in an area rich in subcutaneous fat, and five measurements were taken, averaging the results. After collection, data were collated and statistically analyzed by grouping according to BMI.

[0132] Experimental results

[0133] Bioelectrical impedance acquisition unit data: For the normal-weight group, the mean body fat percentage was 22.5%, with a standard deviation of 3.2%, and the mean phase angle was 5.4°, with a standard deviation of 0.4°. For the overweight group, the mean body fat percentage was 28.8%, with a standard deviation of 3.8%, and the mean phase angle was 4.9°, with a standard deviation of 0.5°. For the obese group, the mean body fat percentage was 35.6%, with a standard deviation of 4.5%, and the mean phase angle was 4.1°, with a standard deviation of 0.6°. At 50 kHz, the mean impedance value for the normal-weight group was 520 Ω, with a standard deviation of 25 Ω; for the overweight group, it was 480 Ω, with a standard deviation of 30 Ω; and for the obese group, it was 440 Ω, with a standard deviation of 35 Ω. At 1 MHz, the mean impedance value for the normal-weight group was 450 Ω, with a standard deviation of 20 Ω; for the overweight group, it was 420 Ω, with a standard deviation of 25 Ω; and for the obese group, it was 390 Ω, with a standard deviation of 30 Ω.

[0134] Ultrasound elastography data: The shear wave velocity of a normal liver (fat content <10%) has a mean of 1.58 m / s, with a standard deviation of 0.1 m / s. When the fat content is 10%-20%, the mean is 1.85 m / s, with a standard deviation of 0.15 m / s. When the fat content is 20%-30%, the mean is 2.1 m / s, with a standard deviation of 0.2 m / s. The shear wave velocity of a normal kidney has a mean of 1.4 m / s, with a standard deviation of 0.1 m / s. It increases to 1.6 m / s, with a standard deviation of 0.15 m / s, when the fat content is 20%-30%. The shear wave velocity of a normal pancreas has a mean of 1.3 m / s, with a standard deviation of 0.1 m / s. It increases to 1.5 m / s, with a standard deviation of 0.15 m / s, when the fat content is 20%-30%.

[0135] Near-infrared spectroscopy unit data: The mean oxygen metabolic rate for the normal-weight group was 1.23 μmol / (g·min), with a standard deviation of 0.15 μmol / (g·min); the mean for the overweight group was 1.05 μmol / (g·min), with a standard deviation of 0.12 μmol / (g·min); and the mean for the obese group was 0.8 μmol / (g·min), with a standard deviation of 0.1 μmol / (g·min). At a depth of 3 mm, the fat signal intensity for the normal-weight group was 0.8 (standard deviation 0.05); for the overweight group it was 0.75 (standard deviation 0.06); and for the obese group it was 0.7 (standard deviation 0.07). At a depth of 8 mm, the fat signal intensity for the normal-weight group was 0.6 (standard deviation 0.04); for the overweight group it was 0.55 (standard deviation 0.05); and for the obese group it was 0.5 (standard deviation 0.06).

[0136] Result Analysis

[0137] Bioelectrical Impedance Acquisition Unit: As BMI increases, body fat increases, altering the current's conductivity and decreasing the phase angle, indicating that the phase angle can reflect changes in body fat content. The obese group exhibited low impedance across different frequency bands, demonstrating that multi-band current excitation can comprehensively reflect fat distribution.

[0138] Ultrasound elastography unit: When liver fat content increases, shear wave velocity and tissue stiffness increase, providing a direct reflection of the extent of liver fat accumulation. Shear wave velocity also increases in the kidneys and pancreas with obesity, but the magnitude of change in elasticity parameters varies across organs, demonstrating the targeted and diverse nature of the test.

[0139] Near-infrared spectroscopy: The obese group showed a decreased oxygen metabolism rate in subcutaneous fat, suggesting that obesity may lead to abnormal subcutaneous fat metabolism. The signal intensities of fat at different subcutaneous depths were decreased in the obese group, with more pronounced differences, demonstrating the effectiveness of the near-infrared spectroscopy unit's depth resolution algorithm.

[0140] Conclusion: This experiment obtained accurate and relevant data through multimodal data acquisition, indicating that the acquisition parameters of each unit are closely related to the degree of obesity, providing a reliable data basis for the visualization system and verifying the effectiveness and feasibility of the multimodal data acquisition method.

[0141] This invention is of great significance in the field of clinical medicine, improving the diagnosis and treatment of obesity-related diseases and safeguarding patient health. Its multimodal data acquisition, precise prediction, visualization, closed-loop health management, and surgical assistance capabilities have revolutionized clinical medicine.

[0142] Accurate diagnosis and assessment

[0143] Comprehensive data collection: The multimodal data acquisition module of the present invention integrates a bioelectrical impedance acquisition unit, an ultrasound elastic imaging unit and a near-infrared spectroscopy unit. 2 Taking a male patient with as an example, the bioelectrical impedance acquisition unit measured the body fat distribution characteristics through multi-band current excitation. The extracted phase angle parameter was 4.0°, which was lower than the normal range, indicating a high fat content. When the ultrasound elastography unit examined the liver, it found that the shear wave velocity was 2.2 m / s, which was higher than the 1.5-1.8 m / s of a normal liver, indicating that fat infiltration of the liver caused increased hardness. The near-infrared spectroscopy unit detected subcutaneous fat oxygen metabolism parameters, showing that the fat oxygen metabolism rate at a depth of 3-8 mm below the skin was 25% lower than the normal level, reflecting abnormal fat metabolism. These multi-dimensional data provide doctors with rich information for a comprehensive understanding of the patient's fat status.

[0144] Accurately predict organ fat deposition: The BMI-driven prediction model improves prediction accuracy by establishing a quantitative mapping model between BMI index and three-dimensional organ fat deposition. For the above patient, the model comprehensively considers factors such as BMI, age, and gender and predicts that the liver fat volume is 130cm 3 , compared with 125 cm measured by subsequent gold standard MRI 3 Compared with traditional single-modality prediction methods, this method effectively reduces prediction bias, enabling doctors to more accurately assess the fat deposition in patients' organs and provide a reliable basis for diagnosis.

[0145] Visualization-assisted diagnosis and treatment

[0146] Intuitively Presenting Fat Distribution and Risk: The 3D visualization module presents organ fat deposition as a 3D model and maps fat density gradients and metabolic risk levels using heat maps. For example, in the 3D liver model of the patient mentioned above, the fat deposits around the portal vein appear as red high-risk areas on the heat map. This intuitively demonstrates high-risk areas of fat distribution, helping doctors quickly locate and assess the severity of the condition and develop targeted treatment plans.

[0147] Predicting disease progression: The dynamic prediction unit simulates the spatiotemporal evolution of organ fat deposition. Through continuous monitoring and model analysis of this patient, it is predicted that the patient's liver fat volume may increase by 10cm in the next 6 months.3 Without timely intervention, fat deposition will worsen, increasing the risk of diseases such as fatty liver. Doctors can adjust treatment strategies in advance based on the predicted results, such as strengthening diet and exercise guidance, or considering drug intervention.

[0148] Personalized health management

[0149] Closed-loop management enables precise intervention: The closed-loop management module of the health management app uses multimodal data to work together. The patient uses a Bluetooth body fat scale, a smart bracelet, and a NIRS sensor for daily monitoring, and the data is uploaded to the app in real time. When the intelligent early warning center detects that the liver fat deposition rate exceeds 1.5cm for three consecutive days, 3 If the patient experiences a high blood sugar level of 20mg / day, a Level 2 alert is triggered. The app automatically sends a text message to the patient, initiating intervention with an AI nutritionist. The AI ​​nutritionist develops a personalized nutrition and exercise plan based on the patient's dietary and exercise habits, lowering the daily carbohydrate intake limit by 20% and recommending three high-intensity interval training sessions per week.

[0150] Improve patient compliance: Through real-time feedback and adjustment of intervention strategies through the APP, patients can intuitively see changes in their health data and the effects of intervention. For example, after adjusting their lifestyle according to the AI ​​nutritionist's advice for one month, the patient lost 2 kg and the liver fat deposition rate dropped to 1.0 cm 3 / day, the APP provides timely encouragement and further optimization suggestions, which improves patients' confidence and compliance with treatment and promotes the continuity of health management.

[0151] Surgical assistance

[0152] Precise Preoperative Planning: The surgical assistance module's 3D fat vascular reconstruction unit, during a preoperative assessment of an obese patient undergoing partial liver resection, clearly demonstrated the characteristics of fatty infiltration around the portal and hepatic veins. The fatty infiltration around the portal vein was found to be up to 6 mm thick, impairing the surgical field of view and the difficulty of the procedure. The virtual resection planning unit, utilizing biomechanical simulation models and finite element analysis, simulated organ deformation and stress distribution after resection of 30% of the right lobe of the liver, predicting the risk of postoperative portal hypertension and providing a scientific basis for surgical planning.

[0153] Safe Intraoperative Navigation: The intraoperative AR navigation unit, based on the Hololens 2, overlays organ fat heat maps with the real surgical field of view. During surgery, if the ultrasonic scalpel tip is within 2mm of the danger triangle, the system automatically triggers an alert, reminding the surgeon to proceed safely. Simultaneously, the ultrasonic lipolysis device power is adjusted based on real-time fat distribution, ensuring precise surgical execution, minimizing complications, and improving surgical success rates.

[0154] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A BMI-driven 3D organ obesity visualization system based on multimodal fusion, characterized by: The method comprises a BMI-driven prediction model for establishing a quantitative mapping model between BMI index and three-dimensional organ fat deposition; The BMI-driven prediction model includes: a multimodal data fusion unit, a model mapping unit and a residual compensation unit; The multimodal data fusion unit is configured with a multimodal data fusion algorithm, which performs spatiotemporal alignment of the multimodal data acquisition module data through transfer learning, dynamically allocates the fusion weight of each modality data based on the attention mechanism, and eliminates motion artifacts based on the wavelet transform through the denoising function; The dynamic mapping network unit integrates a BMI-organ fat three-dimensional mapping model based on a hybrid architecture of Transformer and graph convolutional neural network to establish a nonlinear mapping relationship between BMI index and organ fat volume; The residual compensation unit uses a deep residual network to perform nonlinear correction on the output of the dynamic mapping network unit to compensate for the prediction deviation caused by individual differences.

2. The BMI-driven 3D organ obesity visualization system based on multimodal fusion according to claim 1, characterized in that: The BMI-driven prediction model is signal-connected to a multimodal data acquisition module, which is used to collect various human body data for the BMI-driven prediction model; The multimodal data acquisition module includes: bioelectrical impedance acquisition unit, ultrasonic elastography unit and near-infrared spectroscopy unit; The bioelectrical impedance acquisition unit is connected to the bioimpedance spectrometer to measure the body fat distribution characteristics through multi-band current excitation and extract the phase angle parameters; The ultrasound elastography unit is connected to a high-frequency ultrasound elastography instrument to detect the elastic parameters of organ fat through shear wave velocity and generate an organ tissue hardness distribution map; The near-infrared spectroscopy unit is connected to the near-infrared spectroscopy sensor, uses a dual-wavelength light source to detect the oxygen metabolism parameters of subcutaneous fat, and separates fat signals at different subcutaneous depths through a depth resolution algorithm.

3. The BMI-driven 3D organ obesity visualization system based on multimodal fusion according to claim 1, characterized in that: The BMI-driven prediction model is signal-connected to a three-dimensional visualization module, which is used to implement real-time dynamic rendering of the BMI-driven prediction model; The 3D visualization module includes: physical rendering unit, heat map mapping unit and dynamic prediction unit; The physical rendering unit is based on a GPU-accelerated parallel rendering pipeline and uses a physical lighting model to perform 3D reconstruction of organ fat deposition; The heat map mapping unit maps the fat density gradient into a color gradient using the HSL color space conversion algorithm and superimposes the metabolic risk level indicator; The dynamic prediction unit integrates an improved convolutional long short-term memory network to simulate the spatiotemporal evolution of organ fat deposition and generate prediction visualization results for future time nodes.

4. The BMI-driven 3D organ obesity visualization system based on multimodal fusion according to claim 1, characterized in that: The system is also designed with a health management app, which includes a closed-loop management module. It uses multimodal data collaboration to achieve full-link management of monitoring, early warning, and intervention, and supports cross-platform operation on HarmonyOS, iOS, and Android. The closed-loop management module includes: wearable device access unit, intelligent warning center and dynamic intervention engine; The wearable device access unit integrates a multi-protocol adaptation layer, supports heterogeneous device data fusion such as Bluetooth body fat scales, smart bracelets, and NIRS sensors, and achieves synchronous collection of multi-source physiological parameters through a timestamp alignment algorithm. The intelligent early warning center is connected to the dynamic prediction unit, triggering a graded early warning strategy when the organ fat deposition rate exceeds a threshold; The dynamic intervention engine includes an AI nutritionist, which adjusts nutritional intake and exercise plans in real time based on a reinforcement learning framework and provides users with medication reminders.

5. The BMI-driven 3D organ obesity visualization system based on multimodal fusion according to claim 4, characterized in that: The system is also designed with a federated learning framework module for multi-center data joint modeling; The multi-center federated learning framework includes: local model update unit, global aggregation unit and privacy protection unit; The local model training unit trains the sub-model of the BMI-driven prediction model locally in each medical institution; The global aggregation unit uses differential privacy technology to perform weighted averaging on the parameters of the sub-models to generate a global shared model; The privacy protection unit uses a homomorphic encryption algorithm to ensure the invisibility of the original data during transmission.

6. The BMI-driven 3D organ obesity visualization system based on multimodal fusion according to claim 1, characterized in that: The system is also designed with a surgical assistance module to provide precise metabolic surgery support; The surgical assistance module includes: fat blood vessel 3D reconstruction unit, virtual resection planning unit and intraoperative AR navigation unit; The fat vascular 3D reconstruction unit generates a topological map of the vascular-fat spatial relationship by distinguishing the fat infiltration characteristics around the portal vein and hepatic vein; The virtual resection simulation unit contains a biomechanical simulation model, calculates organ deformation parameters through finite element analysis, and drives the 3D visualization module in real time to display the predicted image results; The intraoperative AR navigation unit uses Hololens2 to superimpose organ fat heat maps with the real surgical field of view, and integrates depth sensors to achieve millimeter-level alignment of virtual and real scenes, allowing real-time navigation to avoid dangerous triangles.

7. The BMI-driven 3D organ obesity visualization system based on multimodal fusion according to claim 1, characterized in that: The multimodal data fusion algorithm configured by the multimodal data fusion unit is as follows: Among them, FatMap represents the fused fat distribution heat map, which is the final output of multimodal fusion result and is used to drive 3D visualization; i is the dynamic weight of the i-th modal data, calculated through the attention mechanism; D i is the original data of the i-th mode, that is, the data transmitted by the multimodal data acquisition module; Normalize() is a data normalization function used to scale data of different dimensions to the [0,1] interval; λ i is the noise reduction coefficient, which is related to the wavelet transform level; Noise() is the noise estimation function, which is based on the energy calculation of the high-frequency component of wavelet decomposition.

8. The BMI-driven 3D organ obesity visualization system based on multimodal fusion according to claim 1, characterized in that: The BMI-organ fat three-dimensional mapping model integrated by the dynamic mapping network unit is as follows: V organ =α·BMI β +γ·Age+δ·Sex+∈·WHR+NN residual (X) V organ It is a quantitative indicator of three-dimensional organ fat deposition, which is used to represent the fat volume or density of a specific organ; α is the BMI basic coefficient, which is used to control the linear effect of BMI on organ fat; β is the BMI nonlinear index, which is used to control the nonlinear effect of BMI on organ fat; γ is the age coefficient, which is used to control the effect of age on organ fat; δ is the sex coefficient, which is used to control the effect of sex on organ fat; ε is the waist-to-hip ratio coefficient, which is used to control the effect of waist-to-hip ratio on organ fat; NN residual (X) is a residual neural network with input X = [HbA1c, insulin, CT value] to compensate for nonlinear effects not captured by the model.

9. A management method for a BMI-driven 3D organ obesity visualization system based on multimodal fusion, applicable to the BMI-driven 3D organ obesity visualization system based on multimodal fusion according to any one of claims 1 to 8, characterized in that: The management method comprises the following steps: S1. Synchronous multimodal data acquisition: Bioimpedance spectrometer, ultrasound elastography, and near-infrared spectroscopy sensors are used to synchronously acquire the subject's biophysical characteristics. S2. Cross-modal feature fusion: Utilize the transfer learning framework to align the feature representations of data from different modalities and dynamically fuse multi-source data through the attention mechanism; S3. Organ fat volume prediction: BMI and fusion features are input into the quantitative mapping model to calculate the three-dimensional fat distribution parameters of each organ; S4. Dynamic visualization: Generate organ fat heat maps based on a physical rendering engine and overlay metabolic risk warning information.

10. The management method of the BMI-driven 3D organ obesity visualization system based on multimodal fusion according to claim 9, characterized in that: The management method further comprises the following steps: S11. Add motion artifact elimination processing during the data acquisition stage and use wavelet transform algorithm to filter out noise interference in bioimpedance measurement; S21. Implement federated learning optimization during the feature fusion process and regularly aggregate the model parameter updates of each medical institution; S31. Enable AR interaction function during visualization and achieve three-dimensional spatial annotation of fat deposition areas through head-mounted display devices; S41. Integrate a real-time feedback mechanism into the health management closed loop and dynamically adjust intervention strategy parameters based on user compliance.

Images